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1  .  INTRODUCTION  AND  EXECUTIVE  SUMMARY 

1.1.  Ov«rvl«w 

This  research  seefcs  to  characterize  the  organization  of  data  in  memory  used  in  ‘schema- 
based*  decision  making.  This  characterization  is  intended  to  help  identify  methods  to  support 
training,  planning,  and  situation  assessment  associated  with  distributed  decision  n^ing.  Of 
particular  interest  are  methods  for  presenting  Information  that  improve  coordination  by  pron>oting 
a  more  uniform  interpretation  of  situations  among  decision  makers. 

Schema-based  decision  making  arises  when  decisions  are  based  primarily  upon  situation 
recognition.  In  such  cases,  people  recognize  that  the  characteristics  of  a  new  situation  and 
decision  task  match  those  of  previously  encountered  situations  and  tasks  sufficiently  well  so  that 
actions  that  worked  in  these  previously  encountered  situations  are  likely  to  work  In  the  new 
situation. 

We  assume  that  people  rely  on  their  menwry  to  recognize  the  relevance  of  old  situations  to 
new  decision  tasks.  Further,  we  assume  that  the  data  in  memory  are  organized  in  a  particular  way 
that  facilitiates  the  information  processing  needed  for  situation  assessment.  If  this  organization  of 
data  can  be  characterized,  then  it  may  be  possible  to  identify  procedures  and  aids  for  training, 
planning,  and  sKuation  assessment  that  take  advantage  of  this  organization.  Such  methods 
should  enable  people  to  learn  to  interpret  ambiguous  situations  more  accurately,  and  should 
reduce  differences  In  situation  interpretations  among  the  members  of  a  distributed  decision 
making  team. 

This  report  describes  research  that  examines  the  organization  of  data  in  memory  during  a 
decision  making  task. 

1.2.  Contribution  to  distributed  decision  making 

Distributed  decision  making  is  a  particuisff  type  of  group  decision  making.  In  distributed 
decision  making  each  member  of  a  group  acts  individually,  making  decisions  in  an  area  of 
responsibility  or  expertise  to  advance  group  objectives.  Each  member  must  consider  not  only 
how  his  decision  may  affect  Ws  own  area  of  responsibility,  but  must  also  consider  how  rt  affects  the 
other  decision  makers  and  the  objectives  of  the  group. 

Coordination  among  distributed  decision  makers  is  supported  by  a  plan  that  specifies  the 
actions  to  be  taken  for  various  possible  situations.  This  plan,  which  may  be  developed  by  the 
decision  makers,  specifies  the  actions  to  be  taken  by  each  decision  maker  in  each  of  these 
situations.  During  the  execution  of  the  plan,  each  decision  maker  evaluates  the  situation  and 
decides  which  of  the  planned  actions  is  appropriate  under  the  plan  or  modifies  the  plan  to 
accommodate  unexpected  circumstances.  Successful  coordination  deperKts  on  a  common 
understanding  of  the  plan  and  interpretation  of  tt)e  situation  among  all  decision  makers,  arxl  an 
ability  of  each  decision  maker  to  accurately  assess  the  impact  of  a  decision  on  the  responsibllHies 
of  others.  Communications  can  help  each  decision  maker  to  interpret  the  situation  the  same  way 
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as  the  others  do,  anticipate  the  actions  of  other  decision  makers,  and  estimate  his  impact  on  the 
responsibilities  of  others. 

Poor  coordination  may  arise  If  different  decision  makers  interpret  situations  differently,  It 
they  fail  to  seek  or  communicate  essential  information,  or  if  they  fail  to  anticipate  the 
consequences  of  their  actions  on  others.  These  problems  may  sometimes  be  caused  by 
differences  in  people's  expectations  about  the  indicators  of  possible  different  situations  arKf  the 
consequences  of  actions.  Such  differences  may  result  from  differences  in  the  n)en)ory  data 
descriptive  of  the  situations. 

The  work  reported  here  is  intended  to  clarify  how  information  used  for  situation 
interpretation  is  organize<j  in  memory.  Knowledge  of  this  memory  organization  can  provide  a 
basis  for  situation  assessment  aids  and  for  planning  and  training  methods  that  help  an  decision 
makers  interpret  situation  data  the  same  way,  help  them  better  anticipate  how  the  decisions  of 
different  team  members  impact  each  other,  and  help  them  to  identify  information  that  should  be 
communicated  and  sought. 

1.3.  Relationship  to  previous  year's  research 

During  the  first  two  years  of  research  in  distributed  decision  making.  Engineering  Research 
Associates  developed  and  tested  a  schema  model  of  information  processing  for  situation 
assessment.  This  model  proposed  that  memory  is  structured  as  a  hierarchy  of  ’fuzzy'  schemata. 
See  Figure  1  -1 .  Each  schema  contained  three  layers:  a  layer  for  identifying  situation  features 
relevant  to  situation  interpretation,  a  second  layer  for  evaluating,  weighting,  and  combining  these 
features  into  an  overall  assessment,  and  a  third  layer  that  specifies  actions  and  inferences 
appropriate  to  situations  represented  by  the  schema.  Data  collected  in  psychology  experiments 
showed  that  the  schema  model  is  able  to  account  for  people's  situation  assessments  under  the 
conditions  of  the  experiments. 

The  research  reported  here  extends  the  previous  research  in  two  different  ways.  First,  it 
tests  the  generality  of  the  schema  model.  Second,  it  examines  the  role  of  the  schema  not  only  in 
situation  assessment,  but  also  in  decision  making. 

The  generality  of  the  schema  model  was  tested  by  varying  task  training  methods.  In  the  first 
year's  experiment,  subjects  were  trained  to  assess  situations  in  a  way  suggested  by  the  schema 
model.  They  were  told  what  situation  features  were  relevant  to  the  situation  assessment,  and 
they  were  presented  with  situation  examples  that  facilitated  their  abstracting  appropriate  feature 
evaluation  and  combination  methods. 

In  the  current  experiments,  subjects  were  not  explicitly  trained  to  recognize  situations. 
Rather,  they  were  given  a  decision  task,  and  were  trained  to  evaluate  the  possible  decision 
alternatives  using  formal  measurement  arxl  mathematical  methods.  It  was  hypothesized  that  with 
experience  the  subjects  would  begin  to  abandon  the  rather  tedious  formal  methods  and  begin  to 
use  methods  based  on  recognition.  That  Is,  the  subjects  would  begin  to  use  features  of 
situations  that  would  enable  them  to  recognize  the  superior  alternative,  and  thereby  permit  them 
to  discontinue  using  the  formal  methods.  The  experimental  conditions  encouraged  this  process. 
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Figure  1*1:  Model  of  fuzzy  schema. 


by  forcing  the  subjects  to  evaluate  the  situation  and  make  their  decisions  very  quickly.  The 
experiments  were  designed  to  document  the  use  of  memory  in  decision  making  and  to 
discriminate  among  alternative  memory  models. 


The  data  on  decision  making  performance  was  intended  to  document  the  roles  played  by 
two  different  decision  strategies:  decision  meting  based  on  recognition,  and  decision  making 
based  on  projecting  the  consequences  of  alternatives.  In  recognition-based  decision  making,  no 
explicit  outcome  calculation  is  necessary.  An  alternative  can  be  selected  because  it  is  known  to 
work  in  the  kind  of  situation  being  encountered.  On  the  other  hand,  Indecision  making  based  on 
projecting  the  consequences  of  the  decision  alternatives,  outcome  calculation  is  always  required. 

1.4.  Summary  of  principal  findings 

The  data  from  the  experiments  support  seven  general  conclusions  about  the  formation  and 
characteristics  of  schemata  which  support  decision  making.  These  conclusions  are: 

1 .  Memory  data  able  to  support  recognition-based  decision  making  will  develop  from 
experience  with  a  decision  task  based  on  outcome  calculation. 

2.  These  data  contain  more  information  than  just  remembered  instances. 

3.  The  structure  of  data  in  these  schemata  are  consistent  with  the  fuzzy  schemata  model 
examined  in  the  earlier  experiments  conducted  at  ERA. 

4.  Memory  reference  structures  that  consist  of  boundaries  between  different  situation  classes 
did  not  contribute  to  decision  making  in  these  experiments. 

5.  An  overall  decision  making  task  may  include  components  of  both  recognition  and  outcome 
calculation. 

6.  Subjects  did  not  notice  or  use  arbitrary  but  highly  diagnostic  indicator/  counterindicator 
features  of  the  best  decision  alternatives. 

7.  Schemata  to  support  recognition  of  complex  situations  may  develop  very  slowfy,  or  may  not 
develop  at  all. 

1.5.  Organization  of  report 

This  report  has  five  additional  sections.  Section  2  briefly  describes  alternative  oxxlels  of 
decision  making,  reviews  previous  work  in  this  area,  and  describes  the  key  issues  examined  in  the 
ERA  research  reported  here.  Section  3  describes  the  tasks  performed  by  subjects  in  the 
experiments,  and  describes  the  specific  form  of  memory  reference  strucfures  investigated  for 
these  particular  decision  tasks.  Sections  4  and  5  describe  the  experiment  design,  methods,  and 
results  for  the  two  major  experiments  investigating  schema-based  decision  making.  Section  six  is 
a  general  discussion  and  summary. 
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2.  MODELS  OF  JUDGMENT  AND  CHOICE 
2.1.  Benefits  from  models 

This  study  investigates  and  models  the  memory  structures  that  form  when  people  gain 
experience  with  an  analytical  decision  task.  Models  of  these  memory  structures  are  potentially 
useful  if  they  provide  a  theoretical  foundation  for  training,  pianning,  and  decision  aiding  methods 
based  on  natural  cognitive  processes.  These  methods  could  enable  novices  to  acquire  expert 
skills  more  rapidly  than  current  methods  do,  and  could  help  novices  to  perceive  the  essential 
elements  of  a  planning  or  decision  task,  enabling  them  to  see  these  tasks  "through  the  eyes  of  an 
expert." 

Larkin  and  Simon  (1987)  demonstr  ated  how  a  model  of  human  cognition  and  memory 
organization  could  support  decision  aid  design.  They  described  how  the  relative  effectiveness  of 
diagrams  or  text  couid  be  understood  in  terms  of  a  production  rule  model  of  knowledge.  In  this 
model,  knowledge  is  stored  as  a  sequence  of  if-then  rules.  Like  Larkin  and  Simon,  we  believe 
that  cognitive  modeis  can  support  a  theory  of  information  presentation,  but  we  do  not  assume  that 
production  rules  are  necessarily  a  good  model  of  memory  organization.  The  experiments 
described  in  this  report  investigate  the  memory  structures  that  develop  in  specific  decision  tasks. 

This  section  briefly  reviews  the  alternative  cognitive  models  that  motivated  our 
investigations.  These  include  decision  making  models,  models  derived  from  the  literature  of 
schemata,  classification,  and  decision  making,  and  the  situation  assessment  model  that  ERA 
recently  developed.  The  concluding  discussion  outlines  the  goals  of  the  current  research  in 
terms  of  this  review. 
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2.2.  Decision  making  models 
2.2.1.  General  model 

Figure  2.1  (Lawson,  1987)  is  a  general  model  of  decision  making.  It  contains  four  stages: 
process,  assess,  evaluate,  and  choose.  In  the  process  stage,  people  convert  perceptual  data 
into  information  useful  for  assessing  a  perceived  situation.  The  situation  assessment,  perfomrred 
in  the  assess  stage,  is  an  estimate  of  the  situation.  In  a  military  setting,  this  estimate  IrKdudes  the 
location  and  identification  of  forces,  possible  hostile  objectives  and  possible  courses  of  action  to 
achieve  these  objectives.  The  next  stage,  evaluation,  is  the  estimate  of  how  the  situation  may 
change.  The  estimate  includes  projections  of  consequences  of  possible  decision  alternatives.  In 
the  final  stage,  choice,  people  compare  the  consequences  of  different  alternatives  and  select  the 
best  one  . 

This  model  encompasses  two  contrasting  models  of  decision  making:  rational  outcome 
calculation,  and  recognition  (or  schema-based)  decision  making. 
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Figure  2-1.  General  model  of  decision  making. 


2.2.2.  Rational  outcome  calculation 

The  outcome  calculation  model  of  decision  making  looks  at  decision  making  as  a  rational 
process  of  explicitly  comparing  options  and  choosing  the  optimal  alternative.  It  emphasizes  the 
third  and  fourth  stages  of  the  general  model.  It  proposes  that,  to  the  extent  possible,  people 
project  the  outcomes  associated  with  each  alternative,  convert  each  of  the  outcomes  to  utilities 
that  reflect  the  desirability  of  each  outcome,  weight  the  importance  of  each  outomce,  estimate  a 
total  utility  for  each  alternative,  and  then  select  the  alternative  with  the  highest  utility. 

Furthermore,  the  model  assumes  that  the  weight  attached  to  an  uncertain  outcome  reflects  the 
probability  of  that  outcome. 

Although  this  model  describes  some  human  decision  making,  as  a  general  tTX)del  it  has  two 
major  difficulties:  it  does  not  address  how  the  alternatives  to  be  evaluated  are  identified;  and  it  is 
inconsistent  with  well-documented  examples  of  human  judgment  and  choice  (Elnhom  and 
Hogarth,  1981). 

Some  of  the  inconsistencies  between  the  rational  outcome  model  and  human  behavior 
suggest  that  sometimes  people  employ  a  fundamentally  different  process  of  decision  making. 
People  have  been  observed,  for  example,  to  ignore  elementary  laws  of  probability  (Tversky  and 
Kahneman,  1983),  and  their  preference  (utility)  for  a  bet  may  depend  on  whether  they  are  being 
asked  which  bet  they  prefer  or  which  they  would  be  willing  to  pay  most  to  play  (Qrether  and  Plott, 
1979).  In  addition,  in  some  field  studies  of  decision  making  experienced  decision  makers  rarely 
consider  more  than  a  single  alternative  (Klein,  1986).  Their  decisions  seem  to  depend  primarily 
on  situation  recognition,  not  on  outcome  projection  and  alternative  evaluation.  Kenneth 
Hammond  vl986)  characterizes  recognition-based  decision  making  as  'intuitive'  decision 
making,  contrasting  it  with  the  more  structured  'analytic'  rational  decision  making. 

2.2.3.  Recognition-based  decision  making 

in  decision  making  b£ised  on  recognition,  people  choose  a  'standard  action  for  a  standard 
situation'.  The  decision  seems  to  follow  directly  from  a  recognition  of  the  type  of  situation  and  a 
recollection  of  what  actions  usually  work  well  in  this  kind  of  situation.  For  example,  one  might 
reason  that  because  a  given  situation  is  very  much  like  situation  X,  then  actions  Y  that  usually  work 
well  in  this  situation  should  be  considered. 

Recognition-based  models  emphasize  the  'process'  and  'assess'  phases  of  the  decision 
process;  evaluation  and  choice  follow  immediately  from  the  assessment.  In  these  phases  people 


characterize  the  decision  task  as  a  kind  of  decision  problem,  select  a  decision  strategy  appropriate 
to  this  kind  of  problem,  and  identify  promising  alternatives.  In  their  1981  review  Einhom  and 
Hogarth  stressed  the  importance  of  these  processes.  They  cited  research  which  demonstrates 
the  importance  of  the  mental  representations  of  the  decision  problem.  Different  representations 
of  the  decision  problem  are  shown  to  result  in  different  decisions  and  judgements. 

Although  people  may  not  be  aware  of  their  decision  process  when  basing  their  decision  on 
situation  recognition,  the  decisions  are  not  random  and  must  reflect  considerable  (if 
subconscious)  information  processing.  We  believe  that  recognition-based  decision  making  is 
based  on  people's  ability  to  classify  situations  and  employs  cognitive  methods  similar  to  those 
used  in  object  classification.  These  methods  are  discussed  below  under  situation  assessment 
models. 


2.2.4.  Hybrid  decision  strategies 

Recognition-based  decision  making  and  rational  outcome  calculation  reflect  poles  of  a 
range  of  decision  making  strategies.  Many  decisions  may  contain  components  of  both  of  these 
modes.  Such  hybrid  strategies  are  examined  by  the  experiments  described  in  this  report. 

Three  factors  influence  the  relative  role  played  by  recognition  versus  outcome  calculation  in 
a  decision.  These  are  knowledge  or  familiarity  with  the  decision  task,  difficulty  of  projecting 
outcomes,  and  importance  of  the  decision.  The  more  familiar  a  person  is  with  a  situation,  the 
larger  the  role  subconscious  classification  will  play  in  his  decision  making.  Difficulty  in  projecting 
outcomes  caused  by  time  or  resource  limitations  or  by  a  large  number  of  situation  uncertainties 
encourages  a  person  to  rely  more  on  the  classification  process.  Finally,  the  less  important  H  Is  that 
the  person  make  the  very  best  decision,  the  more  likely  the  person  is  to  rely  on  subconscious 
classification. 

2.3.  Situation  assessment  models 

Two  general  types  of  assessment  models  are  important  to  recognition-based  decision 
making:  classification  models  and  schema  models.  These  two  types  of  models,  described  in 
section  2.3.1  and  2.3.2,  are  closely  related,  but  are  tailored  to  explain  different  types  of  cogrtitive 
tasks.  Classification  has  focused  primarily  on  categorization  while  schema  have  been  more 
concerned  with  understanding.  The  model  developed  and  tested  by  ERA  contains  aspects  of 
both  the  classification  and  schema  models.  It  is  described  in  section  2.3.3. 

2.3.1.  Classification  models 

The  classification  model  considered  here  is  the  "probablistic  feature  model'  (Smith  and 
Medin,  1981).  This  model  assumes  that  categories  are  defined  In  terms  of  a  set  of  features,  but 
that  there  need  not  be  any  necessary  or  sufficient  feature  sets.  Rather,  classification  arises  from  a 
weighted  sum  of  features,  with  each  feature  weighted  according  to  Its  importance  in  the  category 
being  considered.  If  the  sum  is  high  enough,  then  the  object  is  assigned  to  the  category; 
otherwise  it  is  not. 


The  role  of  characteristic  properties  or  features  has  been  explored  extensively  in  the 
classification  resesrch.  A  feature  that  Is  more  diagnostic  of  a  certain  class  (able  to  discriminate 
anK>ng  different  classes)  Is  judged  to  be  more  typical  and  more  representative  of  a  class  even 
though  It  may  not  be  a  necessary  feature.  For  example,  "ability  to  fly"  would  bo  judged  more 
representative  arxt  more  typical  of  birds  than  would  "animate"  even  though  all  birds  are  alive  and 
not  all  birds  fly. 

Typicality  influences  judgments  and  decisions.  People  judge  typical  members  of  a  concept 
more  quickly  (Rips,  Shobon,  and  Smtth,1973;  Rosch,1973:  Smith,1978)  and  more  accurately 
(Mervis,1980).  Representativeness  is  not  based  on  a  conscious  mathematical  calculation,  but 
rather  on  a  judgment  of  feature-based  similarity.  Representativeness  was  found  to  be  more 
consistent  with  predicted  outcomes,  while  prior  probabilities  and  expected  predictive  accuracy 
tended  to  be  ignored  (Kahneman  and  Tversky,  1973).  Representativeness  can  strongly 
influence  probability  judgment,  and  can  cause  people  to  estimate  probabilities  that  violate 
elementary  laws  of  probability  (Tversky  and  Kahneman,  1983) 

Classification  may  depend  not  only  on  which  features  are  present,  but  also  on  how  much 
each  feature  is  present.  Some  features  can  be  considered  to  be  present  to  a  degree.  Thus,  the 
color  "yellow"  may  be  a  feature  of  malaria,  with  different  shades  of  yellow  being  associated  with 
different  degrees  of  the  malaria  feature  "yellow".  Classification  of  objects  with  such  features  have 
been  modeled  using  fuzzy  sets  (Zimmermann  and  Zysno,  1980).  The  fuzzy  dassiflcation  of 
object,  interpreted  as  the  degree  to  which  an  object  belongs  in  a  class,  can  be  related  to  the 
extent  to  which  the  characteristics  of  each  of  the  object's  features  resembles  the  characteristics  of 
features  expected  of  objects  in  the  class. 

The  processes  by  which  features  are  abstracted  and  organized  into  mental  staictures 
during  category  learning  are  not  yet  fully  understood  (Smith  and  Medin,  1981 ).  It  is  unclear  which 
features  will  be  abstracted  and  how  they  will  be  cognitively  represented.  Lewis  ar>d  Anderson 
(1985)  investigated  the  development  of  categories  associated  with  actions  chosen  In  solving 
geometry  problems.  They  hypothesized  that  features  associated  with  an  action  category  would 
be  used  in  selecting  an  action.  Their  surface  features  were  like  cues  paired  with  specific  actions. 
Subjects  who  reported  noticing  the  pairings  used  the  cues.  Others  did  not.  Lewis  and  Anderson 
found  no  evidence  for  subconscious  learning  of  the  surface  feertures.  They  found  that 
"necessary"  features  contributed  no  more  to  cateogy  learning  than  did  "incidental"  features 
(although  the  subjects  may  not  have  known  which  features  were  necessary). 

According  to  Murphy  and  Medin  (1985),  subjects  need  a  theory  or  approach  to  identify 
which  features  matter  in  dassHicatlon.  Such  a  theory  would  dictate  to  which  features  people 
should  attend,  and  may  reflect  the  purpose  of  a  category.  In  the  Lewis  and  Anderson  experiment, 
subjects  who  did  not  use  the  cues  were  basing  their  decisions  on  something(s)  other  than  what 
Lewis  and  Anderson  defined  as  the  features. 

While  Lewis  and  Anderson  did  not  find  support  for  actions  being  derived  from  a  feature- 
based  dassication  process,  there  is  research  that  suggests  the  importance  of  features  and 
classification  to  action  selection  when  K  is  impossible  to  project  arnf  compare  all  possible 
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outcomes.  Chase  and  Simon  (1973)  studied  the  differences  between  beginning.  advar>ced,  and 
expert  chess  piayers.  They  found  that  expert  piayers  consider  fewer  altemaiives  than  iess  expert 
piayers  and  that  those  alternatives  considered  by  experts  tend  to  be  the  better  ones.  Thus,  in 
chess,  where  compiexity  preciudes  projecting  all  possible  outcontes,  the  expert's  knowledge 
enables  him  to  explore  a  limited  number  of  possibilities.  'Good  moves  just  seem  to  come  to 
mind." 


There  is  some  evidence  that  the  actions  ("good  moves")  came  to  mind  because  experts 
had  knowledge  staictures  for  categorizing  situations  and  associating  types  of  situations  with 
possible  actions.  While  there  were  no  differences  in  their  abilities  to  remember  other 
information,  better  chess  players  had  better  memories  for  chess  playing  information.  The  study 
found  that  experts  tended  to  process  larger  'chunks'  of  infomoation  and  the  infonnation  that  was 
chunked  together  was  related.  The  errors  that  experts  tended  to  make  In  replicating  chess 
positions  did  not  change  the  functional  relationships  between  the  pieces.  These  observations 
are  consistent  with  expert  chess  players'  knowledge  representations  being  categories  whose 
characteristic  features  are  based  upon  the  functional  relationships  between  pieces.  The  experts' 
ability  to  replicate  chess  positions  can  be  attributed  to  the  Information  in  these  categories.  Thus 
the  expert's  short  term  ntemory  for  chess  needs  only  to  remember  the  category  and  how  a 
particular  situation  is  a  variant  of  that  category. 

2.3.2.  Schemata 

Schemata  are  mental  structures  that  organize  knowledge  about  situations  and  associated 
actions.  They  mediate  understanding  of  social  situations  and  stories  and  link  situations  to  actions 
(Abeison,  1981;  Bower,  Black, and  Turner, 1979;  Graesser,  Woll,  Kowalski,  and  Smith,  1980; 
Rumelhart,  1 981 ).  They  are  similar  to  structures  proposed  for  classification.  Each  schema 
contains  a  number  of  embedded  schema,  which  function  like  the  features  used  to  classify 
objects. 

Schemata  enable  observed  events  to  be  remembered  by  "chunking"  information. 
According  to  the  script  pointer  plus  tag  hypothesis  (Graesser,  Woll,  Kowalski,  arxl  SmKh,  1980), 
observed  events  are  stored  in  memory  by  partitioning  into  typical  and  atypical  parts.  The  typical 
p  ^rt  is  integrated  into  existing  schemata,  so  that  explicitly  stated  and  inferred  actions  become 
"interrelated  as  a  whole".  In  contrast,  a  separate  and  distinct  memory  tract  is  constructed  for  each 
atypical  action. 

Pennington  and  Hastie  (1986)  showed  a  very  interesting  use  of  schemata  in  evaluating 
evidence  in  a  complex  decision  making  environment,  juror  deliberation.  It  appears  as  If  jurors 
evaluated  evidence  by  matching  the  evidence  presented  with  the  evidence  that  would  be 
expected  were  various  possible  crime  scenarios  true.  The  jurors  seemed  to  use  their  knowledge 
of  human  behavior  to  construct  these  different  scenarios  (scripts)  that  correspond  to  different 
verdict  categories.  The  verdict  judgement  may  be  reached  by  finding  that  crime  scenario  which  is 
most  consistent  with  the  evidence. 
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2.3.3.  ERA  situation  assessment  model 

Engineering  Research  Associates  developed  and  tested  a  model  of  situation  assessment 
drawn  from  classiflcation .  schemata,  and  fuzzy  set  research.  Details  of  this  model  and  of  the 
associated  experiments  are  provided  in  previous  technical  reports  (Noble  and  Truelove,  1985; 
Noble,  Boehm-Davis,  and  Grosz,  1986).  The  foltowing  paragraphs  briefly  describe  the  model  arrd 
experiments. 

Figure  2-2  summarizes  the  proposed  memory  data  and  information  processing  which 
subjects  use  to  estimate  the  severity  of  an  "all-out  attack"  based  on  the  physical  characteristics  of 
a  picture  of  this  attack. 

The  memory  structure  is  a  three  layered  schema.  The  first  layer  defines  the  features  that  are 
relevant  to  situation  assessment.  In  this  case,  these  are  the  features  relevant  to  assessing  the 
severity  of  the  atta^ '  There  are  four  features:  the  number  of  attacking  ships,  the  number  of 
attacking  aircraft,  the  number  of  attacking  submarines,  and  the  geographical  deployment  of  these 
submarines.  The  second  layer  contains  feature  evaluation  data  (upper  right  quadrant  of  the 
figure).  These  data  relate  the  physical  characteristics  of  the  feature  to  a  subjective  estimate  of 
how  characteristic  such  a  feature  is  of  the  situations  being  modeled  by  the  schema.  In  figure  2-2, 
for  example,  the  feature  evaluation  data  for  the  feature  "surface  threat"  relate  different  numbers  of 
surface  ships  to  a  subjective  assessment  of  how  characteristic  that  number  of  ships  is  of  a  very 
strong  all-out  attack. 

Figure  2-2  summarizes  the  three  most  important  information  processing  steps  for  situation 
assessment.  Additional  steps  are  described  in  the  ERA  technical  reports.  First,  the  relevant 
features  are  extracted.  Second,  each  observed  feature  is  scored  using  the  feature  evaluation 
data.  Third,  the  feature  scores  are  used  to  compute  an  overall  all-out  attack  score.  In  this 
example,  this  overall  score  is  the  geometric  mean  of  the  feature  scores. 

This  model  was  tested  in  a  number  of  psychology  experiments.  Under  the  conditions  of 
these  experiments  it  accounts  very  well  for  subjects'  overall  assessments  in  terms  of  the  physical 
characteristics  of  situation  features  and  subjects'  subjective  feature  evaluations. 

Because  the  model  worked  so  well  in  these  earlier  experiments,  H  may  prove  useful  in 
modeling  other  types  of  situation  assessment  tasks.  Before  concluding  that  this  is  the  case, 
experiments  rerrKiving  the  most  important  limitations  of  the  previous  experiment  need  to  be 
conducted.  These  limitations  are: 

1 .  Subjects  were  trained  "schema-like."  They  were  trained  in  a  way  that  encourages  a  memory 
organization  like  that  of  the  schema  model.  They  were  presented  with  a  sequence  of 
pictures,  organized  around  a  "perfect  ten"  prototype.  For  each  example  they  were  told  an 
experts'  assessment  and  the  reasons  for  this  assessment  in  terms  of  features  (air,  surface, 
and  subsurface  threat  numbers  and  deployment).  They  were  not  told  about  the  nuxlel, 
however,  nor  were  they  given  numerical  featajre  evaluation  data. 

2.  The  experiment  task  was  confined  to  situation  assessment.  Subjects  were  not  evaluating 
the  situation  for  the  purpose  of  selecting  among  alternatives. 
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2.4.  Testing  the  model  further 


The  experiments  described  below  test  the  generality  of  the  schema  nxidel  by  removing 
both  of  these  limitations.  These  experiments  examine  the  menK)ry  organization  that  arises  as  a 
result  of  experience  with  an  analytical  decision  task.  The  experiments  address  these  questions. 

If  an  individual  is  trained  to  make  decisions  based  on  a  complex  outcome  calculation 
and  rule,  will  he  continue  to  calculate  outcomes  forever?  Or  will  he  eventually,  as  he 
becomes  expert,  abstract  knowledge  that  replaces  part  or  all  of  the  explicit  c^culation? 

If  knowledge  that  helps  with  the  task  is  abstracted  ,  then  what  is  the  nature  of  this 
knowledge,  how  does  it  relate  to  the  outcome  oriented  procedural  knowledge,  and 
how  is  it  organized  in  n>emory? 

What  do  individuals  do  when  it  is  impossible  to  use  outcome  calculation  because  of 
time  constraints  or  resource  constraints? 

In  terms  of  the  situation  assessment  models  discussed  previously,  these  questions  are: 

1 .  If  people  are  trained  in  a  decision  making  task  according  to  the  rational  outcome  calculation 
model,  will  they,  with  experience,  evolve  toward  a  recognition-based  approach? 

2.  If  recognition-based  decision  making  appears,  then  what  kind  of  memory  structures  will 
arise?  How  will  these  memory  structures  be  related  to  specific  instances  seen  In  training? 
Will  they  be  the  beginning  of  production  rules,  with  specific  situation  indicator/ 
counterindicators  pointing  to  an  alternative?  To  what  extent  will  they  resemble  the  fuzzy 
schema  modeled  in  the  earlier  experiments? 

3.  How  will  the  recognition  based  decision  making  relate  to  the  outcome  calculation  decision 
making  taught  during  training?  Will  the  recognition-based  decision  making  completely 
displace  the  outcome  calculation  decision  making,  or  will  pieces  of  recognition-based 
decision  making  become  integrated  with  pieces  of  outcome  oriented  decision  making? 


3.  TWO  TASKS  FOR  EXAMINING  THE  ROLE  OF  SCHEMATA  IN  DECISION 
MAKING 

3.1.  Introduction 

Two  experiments  were  performed  to  investigate  the  role  of  schemata  in  decision  making.  In 
tx)th  of  these  experiments  subjects  were  trained  to  select  an  alternative  using  rational  analytical 
methods.  They  were  not  told  that  alternatives  could  be  selected  by  situation  recognition,  but 
were  motivated  by  the  complexity  of  the  analytical  method  and  time  pressures  to  adopt 
recognition-based  methods. 

The  experiments,  which  are  described  in  detail  in  sections  4  and  5,  are  designed  to  reveal 
the  role  played  by  memory  when  subjects  select  among  alternatives  in  these  decision  tasks. 

There  are  several  different  memory  rTx>dels  possible.  This  section  describes  these  memory 
models  in  the  context  of  the  specific  decision  tasks  performed  in  the  experiments. 

3.2.  Task  1 

This  task  examined  the  possible  memory  structures  which  support  relatively  simple 
judgments  embedded  within  a  more  complex  decision  t£isk. 

3.2.1.  Task  description 

Each  subject  plays  the  role  of  a  Battle  Group  commarKler  who  is  facing  a  hostile  barrier.  He 
is  presented  with  a  situation  picture  (Figure  3-1 ).  His  Battle  Group  is  located  at  the  "X".  The  ship 
symbols  denotes  the  locations  of  hostile  ships,  and  the  irregular  cross  hatched  shapes  Indicate 
the  submarine  patrol  areas.  The  subject  must  choose  whether  to  traverse  the  barrier  along  the 
straight  path,  traverse  it  along  the  curved  path,  or  not  traverse  it  at  an,  staying  where  he  is.  He  is 
given  a  procedure  and  simple  rule  for  his  decision: 

1 .  Estimate  the  number  of  hits  your  Battle  Group  will  receive  from  the  hostile  forces  along  each 
path. 

2.  Pick  the  path  along  which  you  receive  the  fewest  hits,  unless  this  number  of  hits  is  nx)re 
than  six.  In  that  case,  stay  at  the  'X.* 

The  subjects  are  provided  with  precise  means  for  calculating  hits  from  the  hostile  ships  and 
submariries.  The  subjects  are  told  that  their  Battle  Group  can  be  attacked  by  enemy  ships  only  at 
the  black  dots.  There  are  three  places  they  can  be  attacked  along  the  straight  path  and  five  places 
they  can  be  attacked  along  the  curved  path.  Their  Battle  Group  travels  the  distance  between 
black  dots  in  one  hour.  Their  Battle  Group  may  be  attacked  by  enemy  submarines  anywhere 
along  the  paths.  The  cornputation  of  hits  from  submarines  has  the  following  steps: 

1  Divide  each  submarine  patrol  area  in  half  vertically  along  its  right-left  center  of  gravity.  If  the 
patrol  area  were  cut  out  and  balanced  on  a  mler,  then  this  center  of  gravity  Is  the  line  where 
the  cutout  would  balance. 

2.  Locate  the  center  of  gravity  for  each  half  of  the  cutout.  This  center  of  gravity  is  the  point 
where  each  half  would  balance  on  the  point  of  a  pencil.  Mark  this  point. 
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3.  Measure  the  distance  between  each  of  these  center  of  gravity  points  and  the  straight  and 
curved  paths.  Each  center  of  gravity  point  which  is  within  the  threat  missile  range  (shown  on 
the  legend  of  the  picture)  can  score  one  hit. 

The  method  for  calculating  hits  from  the  hostile  ships  is  somewhat  more  complicated.  The 
steps  for  this  computation  are; 

1 .  Consider  each  path  in  turn. 

2.  When  you  select  a  path,  the  hostile  ships  will  start  to  move  into  positions  where  they  can 
attack.  Fortunately,  the  hostile  ships  are  slow.  In  one  hour  they  can  move  only  the 
'maximum  ship  movement'  (shown  on  the  legend). 

3.  If  after  the  first  hour  the  ship  can  move  to  within  the  threat  missile  range  of  the  first  dot  on 
the  path,  then  that  ship  can  score  a  hit.  Determine  the  possible  positions  of  enemy  ships 
after  one  hour  to  determine  the  number  of  hits,  if  any. 

4.  After  scoring  one  hit,  a  ship  may  change  direction  and  try  to  move  within  the  threat  missile 
range  of  the  next  black  dot  along  the  path. 

5.  Estimate  the  positions  of  ships  after  the  second  hour.  Those  ships  within  the  threat  missile 
range  of  the  second  dot  can  score  a  hit. 

6 .  Repeat  the  process  for  all  successive  dots  along  the  path. 

To  help  them  with  their  measuring  the  subjects  are  given  two  paper  cutouts:  a  circle  whose 
radius  is  the  threat  missile  range,  and  a  ruler  with  divisions  marked  in  units  of  maximum  hourly 
enemy  ship  movement.  Figure  3-2  shows  the  movemente  and  measurements  needed  to 
calculate  the  hits  from  hostile  submarines  and  ships.  Estimating  the  hits  from  a  hostile  ship  by 
projecting  its  future  positions  was  intended  to  represent  scenario  projection  in  decision  making. 

In  order  to  encourage  non-analytic  estimation  methods,  this  process  was  made  intentionally 
complex.  The  ability  of  hostile  ships  to  change  direction  and  follow  the  subject's  Battle  Group 
could  be  confusing,  for  the  hostile  ships  can  sometimes  score  two  hits  by  moving  initially  to  an 
area  between  two  hit  spots  rather  than  moving  directly  toward  one  of  them. 

It  Is  not  actually  necessary  to  consider  the  different  movements  of  hostile  ships  in  order  to 
estimate  the  number  of  hits  from  that  ship.  It  is  possible  to  estimate  these  hits  entirely  from  the 
initial  position  of  the  ships.  Figure  3-3  shows  a  set  of  contour  range  circles  for  Experiment  1  that 
mark  the  areas  of  initial  positions  from  which  hostile  ships  can  score  a  hit  at  various  hit  points  (the 
large  dots  along  the  paths).  A  ship  whose  initial  position  Is  in  one  of  these  circles  can  score  a  hit  at 
the  hit  point  at  the  center  of  the  circle.  If  a  ship  is  located  in  two  circles,  H  can  score  two  hits.  If  It  is 
in  three  circles,  however,  it  can  still  score  only  two  hits  because  ofx:e  a  ship  starts  nwving  toward 
the  first  circle  it  can  no  longer  reach  ttie  tNrd  one  in  time  to  score  a  hit. 

The  subjects  were  not  shown  those  circles,  nor  was  there  any  hint  given  in  the  instructions 
or  training  that  such  circles  exist. 
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Barrier  situation  picture  showing  evaluation  of  ship  and  submarine  hits  (Experiment  1) 


Figure  3-3:  Contour  circles  showing 
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3.2.2.  Possible  memory  structures  for  estimating  ship  hits 

Several  different  memory  structures  which  could  help  people  estimate  whether  the  straight 
path,  the  curved  path,  or  the  stay  option  is  the  best  alternative  might  develop  from  experience 
with  this  task.  These  memory  structures  are:  (1)  remembered  ship  locations  and  associated 
number  of  hits;  (2)  ship  schemata  consisting  of  remembered  locations  and  hits  plus  data  for 
evaluating  the  significance  of  location  changes;  (3)  contour  lines  separating  areas  from  which  a 
ship  can  score  zero,  one,  or  two  hits;  and  (4)  indicator/  counterindicator  features  that  are  linked  to 
promising  alternatives.  The  first  three  memory  structures  could  help  people  estimate  the  number 
of  hits  from  each  ship.  These  estimates  must  then  be  summed  for  each  path  in  order  to  determine 
the  best  alternative.  The  last  option  could  help  them  to  predict  the  best  alternative  without  having 
to  estimate  the  total  number  of  hits  along  each  path. 

Remembered  instances  only 

When  people  begin  learning  this  task,  they  need  to  project  the  movements  of  the  hostile 
ships  in  order  to  estimate  the  number  of  hits  by  that  ship.  As  they  gain  experience,  however,  they 
may  begin  to  recognize  ships  previously  seen  at  particular  locations.  If  they  can  remember  how 
many  hits  ships  at  these  locations  scored,  then  they  can  can  estimate  the  hits  from  those  ships 
from  this  memory.  They  do  not  have  to  actually  measure  the  movements  of  these  ships. 

The  "remembered  instances  only’  hypothesis  assumes  that  a  person  remembers  only  the 
number  of  hits  from  ships  at  various  positions,  and  does  not  know  how  to  modify  Ws  estimate  for 
ships  at  locations  somewhat  displaced  from  these  remembered  locations.  A  person  remembering 
nothing  more  thawi  the  iocations  and  hit  scores  of  previously  seen  ships  loses  his  ability  to 
estimate  the  number  of  hits  when  ships  are  displaced  from  the  locations  of  previously  seen  ships. 
Furthermore,  he  does  not  know  how  to  adjust  his  estimate  as  a  function  of  the  direction  or 
magnitude  of  the  displacement. 

Fuzzy  set  schema  for  individual  hits 


The  schema  for  Individual  hits  generalizes  the  remembered  instances,  enabling  people  to 
adjust  their  ship  hit  estimates  as  a  function  of  the  direction  and  magnitude  of  the  displacement. 
The  schema  for  an  individual  ship  hit  has  the  same  general  structure  and  contains  the  same  types 
of  information  as  the  fuzzy  set  schemata  described  in  the  earlier  ERA  reports.  In  this  decision  task 
there  would  be  a  different  schema  for  each  remembered  ship  location  relative  to  each  relevant  hit 
point.  There  are  many  very  similar  schemata  Each  differs  from  others  by  only  two  parameters. 

The  schema  enables  a  person  to  estimate  from  visual  inspection  whether  a  new  ship  will  be  able  to 
score  a  hit  on  the  relevant  hit  point.  Figure  3-4  iilustrates  a  possible  schema  for  an  individual  ship. 
The  three  layers  of  the  individual  ship  hit  schema  are: 

1  Feature  identification  layer.  This  iayer  identifies  the  features  that  are  relevant  for  estimating 
’he  number  of  hits  from  a  hostile  ship  dose  to  the  location  of  the  remembered  ship  which  is 
responsible  for  this  schema.  These  features  are  derived  from  the  relative  polar  coordinates 
of  the  locations  of  the  new  and  remembered  ships.  One  feature  is  the  angular 
displacement  of  the  new  ship  relative  to  the  remembered  ship.  The  second  feature  is  the 
radial  distance  between  new  and  remembered  locations  divided  by  the  radial  distance 
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ADJUSTMENT  TO  ESTIMATED 
NUMBER  OF  HITS 


Schema  Features  (used  for  identification): 


R:  radial  displacement 
B:  angular  displacement 


Feature  evaluation  data  for  features  R  and  O 
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Figure  3>4:  Schema  for  Individual  ship  (Experiment  1). 
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between  the  hit  point  and  the  iocation  of  the  remembered  ship.  The  poiar  coordinates  are 
centered  on  the  ship  hit  point. 

2.  Feature  evaluation  layer.  This  layer  contains  the  data  for  evaluetting  the  significance  of 
displacements  between  the  new  ship  and  the  remembered  ship.  It  contains  two  feature 
evaluation  curves:  one  for  the  relative  angular  displacement  and  one  for  the  relative  radial 
displacement.  Figure  3-4  shows  the  evaluation  curves.  The  estimated  number  of  hits  is 
invariant  with  respect  to  angular  displacement,  but  can  change  with  radial  displacement. 

3.  Action  arxf  inference  layer.  This  layer  specifies  whether  the  new  ship  will  score  a  hH  on  the 
hK  point  that  serves  as  the  origin  of  the  feature  evaluation  polar  coordinates. 

Because  this  is  a  ’fuzzy’  schema,  the  feature  evaluation  and  inference  data  are  fuzzy  set 
curves.  For  the  example  of  figure  3-4,  the  feature  evaluation  data  for  radial  displacement  shows 
no  increase  for  new  ships  closer  to  the  hit  point  and  a  gradual  reduction  to  zero  for  ships  further 
from  the  hit  point.  The  steepness  of  this  curve  represents  how  fuzzy  the  hit-no  hit  transition  point 
is  to  the  person  with  this  schema.  The  midway  point  (adjustment  of  -.5)  represents  the  point  at 
which  this  person  would  be  equally  unsure  whether  or  not  the  ship  can  score  a  hit.  Note  that  in 
this  example  the  functional  significance  of  a  displacement  cannot  be  inferred  solely  from  Its 
magnitude  .  Very  large  angular  displacements  make  no  difference.  Relatively  small  outward  radial 
displacements  can  have  a  large  effect. 

Contour  lines  separating  areas  of  zero,  one,  or  two  hits 


A  third  type  of  memory  reference  structure  are  contour  decision  curves  separating  the  areas 
of  the  picture  from  which  a  ship  can  get  zero,  one,  or  two  hits.  Such  contour  plots  are  often  the 
basis  for  computer  classification  algorithms.  These  curves  are  very  complex  in  this  experiment, 
being  formed  from  the  arc  segments  of  the  circles  shown  in  Figure  3-3.  The  curves  are  much 
simpler  in  the  task  2  experiment,  and  are  shown  and  discussed  further  there. 
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3.2.3.  Feature  Indicator/counter-indicators  for  alternative  selection 


According  to  the  ACT  theory  explained  in  Lewis  and  Anderson  (1985)  as  people  gain 
experience  with  a  decision  task  they  may  learn  to  associate  features  of  the  situation  with  preferred 
alternatives.  Experienced  people  can  use  these  features  to  make  decisions. 


In  the  training  pictures  for  this  decision  task,  special  features  were  associated  with  preferred 
altematives.  Two  ships  side  by  side  to  the  right  of  the  upper  portion  of  the  straight  path  always 
indicated  that  the  curved  path  Is  best.  A  single  ship  outside  of  the  curved  path  always  indicated 
that  the  straight  path  is  best.  Intersecting  submarirre  patrol  areas  always  indicated  that  the  stay 
option  is  best. 


People  who  learned  to  use  these  features  could  very  easily  select  the  preferred  altemative, 
for  the  best  alternative  is  unambiguously  associated  with  the  best  option. 
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3.3.  Task  2 

This  task  was  designed  to  examine  three  issues:  the  use  of  contour  tines  for  estimating  hits 
from  individual  ships,  the  relationship  between  individual  ship  hit  estimates  and  alternative 
selection,  and  the  existence  of  wholistic  path  or  choice  schemata. 

3.3.1.  Problem  description 


Task  2  is  very  similar  to  the  first  decision  task.  Each  subject  again  plays  the  role  of  a  Battle 
Group  commander  who  is  facing  a  hostile  barrier.  In  this  task  he  is  presented  with  a  situation 
picture  which  shows  the  two  paths  and  the  position  of  hostile  ships  (Figure  3-5).  His  Battle  Group 
is  located  at  the  'X.'  Once  again  the  subject  must  decide  whether  to  traverse  the  barrier  along  the 
straight  path,  traverse  it  along  the  curved  path,  or  stay  where  he  is.  He  is  told  to  select  the  path 
where  he  receives  the  fewest  hits,  unless  that  number  is  more  than  four.  In  that  case,  he  should 
stay  where  he  is. 

The  procedure  for  calculating  ship  hits  is  simpler  in  this  task.  A  hostile  ship  can  strike 
anywhere  along  the  path.  As  the  subject's  Battle  Group  moves  along  the  path,  the  hostile  ships 
move  toward  the  path.  When  the  hostile  ship  is  at  the  closest  point  of  approach  with  respect  to 
the  subjecf  s  ship  (both  are  on  a  line  drawn  perpendicular  to  the  path  being  traversed)  then  the 
subject  calculates  the  number  of  hHs  by  measuring  the  distance  between  the  hostile  ship  and  the 
path.  Ships  within  the  "two  hit"  distance  of  the  line  score  two  hits;  those  within  the  "one  hit* 
distance  score  one  hit. 


3.3.2.  Possible  memory  structures  for  estimating  ship  hits 

Three  memory  structures  for  estimating  ship  hits  are  considered:  remembered  instances 
only,  ship  schema,  and  contour  plots. 

Remembered  instances  only 
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People  remember  the  locations  of  the  ships  previously  seen  and  the  number  of  hits  scored 
from  ships  at  these  locations.  Ships  close  to  the  locations  of  these  remembered  instances  are 
assumed  to  score  the  same  number  of  hits  as  the  remembered  instance  did.  Confidence  In  this 
estimate  falls  as  the  distance  between  the  new  ship  and  remembered  ship  inaeases.  This  loss  of 
confidence  is  a  function  only  of  the  magnitude,  but  not  the  direction,  of  the  displacement. 

Fuzzy  set  schema  for  Individual  hits 


These  fuzzy  set  schema  are  similar  to  the  ones  described  for  task  1 .  There  are  many  of 
these  schema,  potentially  one  for  each  of  the  locations  of  the  remembered  sNp  locations.  Each 
of  the  schemata  has  three  layers:  one  for  identifying  relevant  features;  one  for  feature  evaluation, 
and  one  for  specifying  actions  and  Inferences. 


Again,  the  schema  is  organized  around  the  location  of  a  remembered  ship.  The  number  of 
hits  specified  by  the  schema  is  the  number  of  hits  at  the  remembered  location  with  adjustments 
specified  by  the  features.  The  two  features  for  each  of  these  schemata  are  coordinates  for  the 
displacements  of  thw  new  ship  relative  to  the  remembered  instance.  One  feature  might  be  the 
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Schema  Features  (used  for  identification): 

X:  horizontal  displacement 
Y:  vertical  displacement 


Feature  evaluation  data  for  features  X  and  Y 


HORIZONTAL  DISPLACEMENT  X  VERTICAL  DISPLACEMENT  Y 

Figure  3-6:  Schema  for  Individual  ship  (Experiment  2). 


relative  displacement  of  the  new  ship  parallel  to  the  subject's  ship  path;  the  other  might  be  the 
relative  displacement  perpendicular  to  this  path.  Figure  3-6  illustrates  one  of  these  ship  hit 
schemata. 


Contour  lines  separating  areas  of  zero,  one,  or  two  hits 

Figure  3-7  shows  the  locations  of  all  of  the  ships  displayed  to  the  subjects  at  any  time  in  any 
training  or  test  picture,  and  shows  the  contour  tines  separating  the  zero,  one,  2Uid  two  hK  eu'eas.  K 
is  possible  that  during  training  subjects  abstract  the  positions  of  these  contour  lines,  and  that 
when  estimating  ship  hits  they  compare  the  positions  of  new  ships  to  the  positions  of  the  contour 
lines. 


This  memory  reference  structure  is  distinctly  different  from  the  individual  ship  hit  schema 
described  previously.  In  the  schema  case,  subjects  are  estimating  the  hits  from  new  ships  by 
comparing  the  location  of  the  new  ship  with  the  remembered  locations  of  previously  seen 
instances.  They  use  the  feature  evaluation  data  to  estimate  the  impact  of  the  displacement  of  the 
new  ship  on  the  number  of  estimated  hits.  In  the  contour  lines  case,  subjects  are  comparing  the 
position  of  the  new  ship  to  the  positions  of  the  contour  lines.  A  ship  that  lies  within  the  inner 
contour  line  is  estimated  capable  of  two  hits;  one  that  lies  between  the  inner  and  outer  contours 
can  score  one  hit,  and  one  that  lies  beyond  the  outer  contour  cannot  score  hits. 

Subjects  basing  their  ship  hit  estimates  totally  on  the  position  of  the  contour  lines  would 
estimate  ship  hits  differently  from  those  using  the  locations  of  remembered  instances.  We 
assume  in  the  former  case  that  the  contours  fonn  during  training  as  a  generalization  of  the  ship  hit 
measurements,  and  that  after  training  tiiey  have  no  memory  of  the  locations  of  the  particular  ships 
that  contributed  to  this  generalization.  In  that  case,  the  subject's  contours  and  contour-based  hit 
estimates  might  have  the  following  characteristics: 

1 .  The  contours  are  'fuzzy."  The  subjects  are  not  certeUn  of  the  exact  location  of  the  transition 
line  from  two  hits  to  one  and  from  one  hit  to  none.  As  the  subjects  gain  more  experience 
these  transition  lines  get  more  precise. 

2.  The  location  of  the  contours  may  be  inaccurate  and  subject  to  biases.  They  may  tend,  for 
example,  to  be  systematically  too  dose  to  the  ship  paths. 

3.  The  accuracy  of  the  subject's  ship  hit  estimates  is  a  function  of  the  accuracy  of  his  contour 
lines.  Subjects  with  more  accurate  contoum  should  estimate  ship  hits  more  accurarately. 
Accuracy  should  depend  only  on  the  accuracy  of  the  contours,  and  not  on  the  proximity  of 
the  ship  to  previously  evaluated  ships. 

4.  Confidence  in  the  number  of  ship  hits  depends  on  distance  from  the  subjecfs  contour 
lines.  Subjects  should  be  more  confident  of  hit  estimates  from  ships  further  from  the  fuzzy 
contours  than  for  ships  dose  to  the  contours.  Confidence  in  ship  hits  for  ships  equidistant 
from  the  contour  should  be  approximately  the  same,  even  if  the  ships  are  on  opposite  sides 
of  the  contour.  For  ships  equidistant  from  the  contours,  confidence  should  not  depend  on 
the  proximity  of  the  ship  to  previously  evaluated  ships. 


3.3.3.  Um  of  Individual  hit  astimatea  In  alternative  evaluation 


The  memory  structures  discussed  so  far  are  useful  for  estimating  the  hits  from  individual 
ships.  These  estimates  can  be  used  to  evaluate  edtematives  in  an  'eyeball  and  count’  process, 
whereby  subjects  would  quickly  estimate  the  number  of  hits  from  each  ship  along  each  path,  sum 
the  hits  along  each  path,  and  select  an  alternative  by  applying  the  decision  rule  taught  in  training. 

3.3.4.  Whollstic  schemata  for  evaluating  and  ranking  paths 

Wholistic  schemata  enable  the  sum  of  ship  hits  along  a  path  to  be  estimated  from  the  overall 
pattern  of  ships.  A  schema  of  the  kind  described  in  the  earlier  ERA  reports  would  be  organized 
around  a  prototype  pattern  associated  with  a  particular  sum  of  individual  ship  hits.  Its  first  layer 
would  identify  a  set  of  features  useful  for  estimating  the  total  number  of  ship  hits  arising  from 
similar  patterns  of  ships.  These  features  would  represent  the  different  dimensions  along  which 
one  pattern  could  vary  from  another.  The  second  layer  would  contain  feature  evaluation  curves 
which  relate  the  impact  of  pattern  displacements  of  varying  magnitudes  along  these  different 
dimensions.  The  third  layer,  for  actions  and  inferences,  would  not  play  a  role  In  this  wholly 
situation  assessment  task. 

Figure  3-8  shows  an  example  of  the  kind  of  schema  that  might  support  alternative  selection 
for  the  problems  presented  to  our  subjects.  This  schema  is  organized  around  a  baseline 
reference  pattern,  which  serves  as  the  organizing  prototype.  This  pattern  consists  of  three  ships 
arranged  in  a  triangular  formation.  Five  possible  schema  features  correspond  to  various  types  of 
displacements  of  the  pattern.  These  features  are: 

1 .  Displacement  of  the  pattern  as  a  whole  perpendicular  to  the  path. 

2 .  Displacement  of  the  pattern  as  a  whole  parallel  to  the  path. 

3.  Three  individual  ship  schema  of  the  type  shown  in  figure  3-6.  These  three  features  show 
the  use  of  embedded  schemata,  whereby  "lower  level"  schemata  become  features  of 
higher  level  ones. 

This  schema  could  be  used  to  estimate  the  number  of  hits  from  a  new  pattern  of  ships  by 
comparing  it  with  the  baseline  prototype  pattern.  The  estimate  would  entail  the  following  four 
steps: 

1 .  The  new  pattern  Is  recognized  as  being  suffidently  similar  to  the  baseline  prototype  to 
warrant  use  of  the  schema. 

2 .  The  number  of  ship  hits  by  the  baseline  pattern  is  recalled. 

3.  Individual  ship  locations  with  respect  to  the  overall  pattern  are  evaluated,  and  adjustments 
to  the  baseline  pattern  score  are  calculated. 

4.  The  two  additional  features,  pattern  horizontal  displacement  and  pattern  vertical 
displacement,  are  evaluated.  These  evaluation  results  are  used  to  further  adjust  the 
number  of  ship  hits  estimated  in  the  previous  step. 


4. 


EXPERIMENT  1 


This  section  describes  the  first  experiment  investigating  the  contribution  of  menx)ry 
reference  structures  to  decision  making.  This  task  was  described  in  section  3.2.  Figure  3-1  is  an 
exampie  of  the  situation  presented  to  subjects.  As  described  in  section  3.2,  each  subject  was 
given  detaiied  anaiytic  procedures  for  caiculating  the  number  of  hits  his  Battle  Group  would 
receive  as  it  traversed  the  barrier  along  the  straight  or  curved  paths.  The  subjects  were  told  to 
select  the  path  that  led  to  the  fewest  hits  unless  tWs  number  of  hits  was  more  than  six.  In  that 
case,  they  were  told  to  stay  where  they  are. 
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4.1.  Issues  and  design  overview 

This  experiment  focuses  on  two  general  issues;  the  type  of  memory  structures  that 
subjects  might  use  to  estimate  the  number  of  hits  from  each  hostile  ship;  and  the  possible  use  of 
Indicator/  counterindicator  features  to  bypass  the  ship  hit  estimation  when  selecting  an 
alternative. 

Although  subjects  were  trained  to  select  the  best  alternative  by  projecting  outcomes,  they 
were  encouraged  by  the  design  of  the  materials  and  by  time  pressure  during  the  testing  to  adopt 
a  decision  strategy  based  on  recognition  of  path  quality.  In  particular,  the  following  aspects  of  the 
design  encouraged  recognition  based  judgment  arxl  decision  making. 

1 .  During  testing  subjects  are  not  provided  with  the  ship  movement  and  threat  range 
measuring  tools  which  were  available  during  training. 

2.  It  is  not  possible  to  make  decisions  based  on  simple  visual  comparisons  of  the  straight  and 
curved  paths.  The  different  shapes  of  the  paths  makes  a  direct  comparison  between  the 
straight  and  curved  path  options  impossible.  In  addition,  the  criteria  for  the  stay  option  is  a 
specific  number  of  hits  along  the  better  path,  a  judgment  not  possible  solely  by  comparing 
the  paths. 

3.  The  subjects  were  pressed  for  time.  A  careful  measurement-based  estimate  took  subjects 
about  five  minutes.  During  testing  they  were  given  only  10  seconds  to  rank  the  options 
(and  an  additional  10  seconds  to  record  their  rankirigs). 

4.  The  procedure  was  tedious.  The  analytical  outcome  projection  required  a  large  number  of 
routine  measurements. 

The  following  section  describes  the  detailed  methodology.  Table  4-1  summarizes  the  test 
procedures  designed  to  address  the  Issues  described  in  section  3.2. 

4.2.  Methods 
4.2.1.  Subjects 


The  subjects  were  twenty-five  undergraduate  students  at  George  Mason  University  virho 
C'  received  either  course  aedit  or  payment  for  their  participation.  Five  of  the  twenty-live  subjects  did 

not  reach  criterion  during  training.  All  twenty-five  subjects  participated  in  the  test  portion  of  the 
•i'i  experiment.  The  responses  of  those  who  did  rrot  reach  criterion  were  excluded  from  any  analysis. 
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ISSUE 


EXPERIMENT  PROCEDURE 


1 .  Subjects  trained  to  calculate  1 . 

outcomes  and  use  rules  will 

develop  memory  reference 
structures  which  enable  them  to 
estimate  ship  hits  quickly  without 
measuring. 

2.  Subjects'  memory  reference  2. 

structures  allow  them  to  accurately 
estimate  ship  hits. 

3.  Subjects  remember  specHIc  3. 

instances,  but  do  not  remember 

other  information  useful  for 
estimating  ship  hits. 


4.  Subjects  use  schemata  for  4. 

estimating  hits  from  individuai 

ships.  Schemata  are  organized 
around  the  location  of  previously 
observed  ships.  They  contain 
data  for  estimating  changes  in  hits 
resulting  from  small  changes  in 
ship  location. 

5.  Subjects  abstract  a  mental  map  or  5. 

set  of  contours  that  they  use  in 
estimating  ship  hits. 


6.  Subjects  notice  arKl/or  use  cues  5. 
(indicator/  counterindicator 
features)  that  are  paired  with  the 
alternatives  and  that  identify  the 
best  alternative. 


Subjects  rank  options  quickly  without  tools. 
The  accuracy  of  these  rankings  is  compared 
with  accuracy  expected  were  subjects 
guessing  at  random. 


Subjects  estimate  hits  from  each  hostile  ship. 
The  accuracy  of  these  estimates  is  compute. 


Subjects  evaluate  hits  from  instances  of 
previously  seen  ships  and  of  ships  that  are 
displacements  of  the  seen  ships.  The 
accuracy  of  hit  estimates  from  previously  seen 
and  new  ships  are  compared. 

Subjects  estimate  hits  from  ships  displaced 
from  previously  seen  ships.  The  accuracy  of 
hit  estimates  as  a  function  of  the  direction  and 
magnitude  of  the  displacement  is  computed. 


Subjects  outline  and  shade  areas  from  where 
ships  can  score  hits.  The  ship  hit  estimates 
predicted  from  their  drawirrgs  are  compared 
with  their  actual  ship  hit  estimates. 


Each  of  three  experimenter-chosen  cues  Is 
paired  with  one  of  the  three  options  In  ail  the 
training  pictures  and  In  the  first  half  of  the  test 
The  cue-alternative  pairing  is  broken  In  the 
second  set  of  test  pictures.  Accuracies  of 
subjects'  path  ranking  on  two  halves  of  test  are 
compared.  In  debriefing,  subjects  are  asked  if 
they  noticed  anything  that  helped  them  in 
making  their  de^ions  in  order  to  determine  if 
they  were  aware  of  the  cues. 


Table  4>1.  Issues  and  experimental  procedures  (Experiment  1). 


4.2.2.  Materials 


Materials  consisted  of  (1 )  path  evaluation  pictures  that  portrayed  hostile  barriers  and 
attemative  path  options,  (2)  individual  ship  pictures  that  showed  individual  hostile  ships  and  the 
path  options,  and  (3)  pictures  which  show  the  two  path  options,  but  do  not  contain  any  ships  or 
submarine  patrol  areas. 

The  path  evaluation  pictures  included  twenty-four  training  pictures  and  twenty-four  test 
pictures.  Pictures  were  arranged  in  four  sets  of  twelve  each.  One  set  was  used  for  initial  training, 
one  set  for  training  to  criteria,  and  two  sets  were  used  in  the  test  portion  of  the  experiment.  The 
path  evaluation  pictures  and  asnwer  sheets  used  in  training  were  hardcopy  booklets.  The  path 
evaluation  pictures  used  in  parts  I,  II,  and  III  of  the  testing.were  slides  projected  for  a  specific 
length  of  time.  Sample  pictures  are  shown  in  Figure  3-1  and  A-1 ,  A-2,  and  A-3  in  Aper>dix  A. 

All  path  evaluation  pictures  are  similar  in  several  respects.  Each  picture  has  the  same  path 
options  (straight  and  curved)  represented  by  the  heavy  dark  lines.  The  dots  along  the  paths 
represent  the  distance  that  the  subjects'  Battle  Group  could  travel  in  one  time  unit.  All  pictures 
have  exactly  four  enemy  ships  and  four  shaded  areas  representing  areas  where  enemy 
submarines  may  be  operating.  The  specific  locations  of  the  enemy  ships  and  submarine  areas 
vary  between  pictures  and  determine  for  each  picture  the  rankings  of  the  options:  take  the 
straight  path;  take  the  curved  path;  or  stay. 

In  addition,  every  picture  has  exactly  one  of  three  indicator/  counterindicator  features.  The 
indicator/  counterindicator  features  are: 

A  single  ship  outside  the  curved  path  as  shown  in  Figure  A-1 ; 

A  pair  of  ships  near  the  end  of  the  straight  path  as  shown  in  Figure  A-2;  and 

Overlapping  submarine  areas  as  shown  In  Figure  A-3. 

In  three  of  the  sets  of  path  evaluation  pictures  (initial  training,  training  to  criterion,  and  the 
first  set  of  test  pictures),  the  indicator/  counterindicator  features  reliably  ir>dlcate  the  best  option. 
In  these  three  sets  the  best  option  for  any  picture  having  a  single  ship  outside  of  the  curved  path 
is  the  straight  path.  Likewise,  if  a  picture  has  a  pair  of  ships  near  the  end  of  the  straight  path,  then 
the  curved  path  Is  best.  For  any  picture  with  overlapping  submarine  areas,  the  stay  option  is  best. 

In  the  fourth  set  of  pictures  the  pairing  of  these  potential  features  with  a  best  option  is 
broken.  In  this  set,  subjects  using  the  indicator/  counterindicator  features  would  never  select  the 
correct  option. 
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Each  set  has  two  pictures  with  each  of  the  following  rankings: 

1st  choice  2nd  choice  3£d-ghQice 


1  straight  path  curved  path 

2  straight  path  stay 

3  curved  path  straight  path 

4  curved  path  stay 

5  stay  straight  path 

6  stay  curved  path 


stay 

curved  path 
stay 

straight  path 
curved  path 
straight  path 


Enemy  ships  were  shown  in  24  locations.  Figure  4- 1  shows  the  locations  of  each  of  the  24 
ships.  Within  each  set  of  pictures  ships  in  each  of  the  first  1 2  locations  are  shown  three  times  and 
ships  in  each  of  the  next  12  locations  are  shown  once  each. 


The  sets  of  pictures  were  designed  to  be  of  comparable  difficulty.  Difficulty  in  selecting  the 
best  option  is  believed  to  be  inversely  related  to  the  difference  in  the  number  of  hits  between  the 
first  and  second  ranked  options.  The  differences  in  the  number  of  hits  between  these  options 
were  matched  as  doseiy  as  possible  over  the  four  sets.  Table  4-2  shows  the  number  of  pictures 
in  each  set  having  each  difference  between  the  first  two  choice  options.  The  set  deemed  slightly 
more  difficult  was  chosen  as  the  set  to  be  used  in  the  training  to  criterion.  The  set  in  which  the 
potential  features  are  inconsistent  with  the  options  had  to  be  the  second  half  of  the  test  set.  The 
other  two  sets  were  randomly  assigned  as  initial  training  and  as  the  first  half  of  the  test. 


Table  4-2:  Number  of  picturea  In  each  set  with  differences  between  first  and 
second  choice  options  of  zero,  one,  or  two  hits  on  Battle  Group. 


There  were  five  sets  of  single  ship  pictures  (sets  A,  B,  C,  D,  and  E)  with  12  pictures  in  each 
( ■'  set.  The  ships  in  set  A  are  in  locations  #1-12  and  were  seen  many  times  by  subjects  in  the  training 

and  testing.  Ships  in  the  pictures  in  sets  B,  C,  D,  and  E  are  in  locations  displaced  from  these 
original  12  locations.  Figure  4-2  shows  the  location  of  each  of  the  original  12  ships  and  their 
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locations  and  contours 
(Experinient  1) 


Composite  of  individual  ship  pictures  shown  (part  IV-Experiment  1). 


displacements.  Ships  in  set  B  are  reflections  of  the  original  12  with  respect  to  the  paths  with 
which  they  are  associated.  Ships  in  set  C  are  displaced  from  the  original  location  in  a  direction 
perpendicular  to  ship  hit  circles.  Ships  in  set  D  are  displaced  in  a  direction  parallel  to  the  ship  hit 
circles.  Finally,  ships  in  set  E  are  displaced  from  ships  in  set  D  in  a  direction  perpendicular  to  the 
ship  hit  circles. 

The  last  type  of  material,  used  in  part  V  of  the  experiment,  showed  only  the  straight  and 
curved  paths  and  did  not  display  ships. 

4.2.3.  Procedure 

Training  procedure 

Each  subject  was  told  that  he  was  a  commander  of  a  Battle  Group  whose  mission  was  to 
penetrate  a  barrier  if  possible.  He  was  responsible  for  deciding  whether  his  Battle  Group  should 
try  to  penetrate  the  barrier  along  the  straight  path,  along  the  curved  path,  or  not  £rt  all.  In  the  last 
case,  the  Battle  Group  would  stay  at  the  ”X*  on  the  situation  diagram.  Subjects  were  told  that  one 
way  to  decide  was  based  upon  determining  the  number  of  hits  that  might  be  received  from  enemy 
ships  and  submarines.  They  were  given  complex  procedures  for  determining  the  number  of  hits 
of  each  type  for  each  path  option.  (These  directions  are  summarized  in  section  3.2.)  In  addition, 
they  were  given  'tools',  a  cardboard  threat  range  drde  and  a  cardboard  mler  marired  in  units  of  the 
distance  that  enemy  ships  travel  per  hour.  Tools  were  used  as  an  aid  for  computing  the  hits  from 
enemy  ships  and  submarines.  Subjects  were  told  that  ’there  may  be  other  ways  to  decide  which 
path  Is  best*  and  that  as  they  became  familiar  with  the  pictures  they  might  notice  things  that  could 
be  helpful  in  making  their  dedsions.  The  procedures  for  calculating  hits  were  purposefully 
complicated  to  motivate  subjects  to  seek  non-computational  methods  of  alternative  selection. 

For  each  training  picture,  subjects  were  told  to  take  a  brief  look  at  the  picture  and  to  record 
their  guess  of  the  ranking  of  the  options.  Following  this  guess  subjects  were  asked  to  use  the 
tools  in  order  to  actually  measure  and  draw  the  ship  movements  and  the  submarine  'center-of 
gravity’  dots.  For  the  first  six  pictures  in  the  initial  training  packet,  they  then  summed  the  hits  for 
each  path,  and  compared  their  picture  with  an  answer  sheet  that  contained  the  correct  ship 
movements  and  submarirre  dots  for  that  picture.  They  conduded  by  recording  the  correct 
rankings  of  the  options  for  the  picture.  For  the  second  six  pictures  in  the  initial  training  booklet, 
their  ’tools’  were  taken  away.  Following  the  quick  initial  guess  subjects  were  asked  to  carefully 
estimate  the  hits  from  ships  and  submarine  areas.  They  then  summed  their  estimated  hits  for 
each  path,  compared  their  results  with  the  answer  sheets’,  and  recorded  the  con-ect  rankings. 

The  answer  sheets  for  the  second  six  pictures  in  this  set  also  showed  the  correct  ship  movements 
and  submarine  dots,  as  well  as  the  number  of  hits  from  ships  and  submarines  and  the  total  number 
of  hits  for  each  path. 

Subjects  were  then  given  a  second  set  of  twelve  training  pictures  and  trained  to  criterion. 
This  training  was  similar  to  the  previous  training.  Subjects  looked  at  the  pictures  briefly  and 
recorded  their  guesses  of  the  ranking  of  the  alternatives.  Then  they  reviewed  the  correct  answer 
on  the  answer  sheet.  The  answer  sheets  for  this  part  did  not  have  the  lines  for  the  ship 
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movements  drawn  nor  were  the  dots  for  the  submarine  areas  drawn.  The  answer  sheets  did 
however  have  the  number  of  hits  from  each  ship  and  submarine  and  the  total  number  of  hHs  alor>g 
each  path.  Subjects  reached  criterion  if  they  correctly  chose  the  first  ranked  option  in  at  least  nine 
of  the  twelve  pictures. 

Test  procedure 

The  test  portion  of  the  experiment  contained  five  parts. 

In  part  I  subjects  ranked  the  options  (straight,  curved,  and  stay)  for  the  24  test  pictures  (sets 
3  and  4).  Each  picture  was  projected  for  10  seconds.  Subjects  then  had  10  additional  seconds  to 
record  their  rankings.  All  of  the  pictures  in  set  3,  in  which  the  indicator/  counterindicator  features 
reliably  indicate  the  correct  option,  were  shown  before  the  pictures  set  4,  in  which  the  indicator/ 
counterindicator  features  always  indicated  an  incorrect  option.  Within  each  set,  the  order  of 
j.resentation  was  randomized. 

In  part  II  of  the  testing,  subjects  were  asked  to  rate  each  of  the  paths  and  the  barrier  (as  a 
whole)  on  a  scale  of  1  to  10.  The  first  12  test  pictures  (set  3)  from  part  I  were  randomized  three 
times  in  this  part  of  the  test.  The  first  time  the  set  was  shown,  subjects  rated  how  good  the  barrier 
would  be  at  blocking  the  Battle  Group  along  the  straight  path.  The  second  time  the  set  was 
shown,  subjects  rated  how  good  the  banier  would  be  at  blocking  the  Battle  Group  along  the 
curved  path.  The  third  time  the  set  was  shown,  subjects  rated  how  good  the  barrier  would  be  at 
blocking  the  Battle  Group  overall.  Each  picture  was  projected  for  five  seconds  each  time  H  was 
shown. 

In  part  III  of  the  testing,  the  same  1 2  test  pictures  were  shown  once  again  in  rarKk>mized 
order.  In  this  part,  subjects  rated  two  statements:  (a.)  ’Enemy  ships  are  near  the  path’  and  (b.) 
"Path  is  patrolled  by  enemy  subs.*  The  statements  were  rated  for  how  characteristic  each 
statement  was  of  each  path  in  each  picture.  Each  picture  was  projected  for  20  seconds  during 
which  time  the  subjects  analyzed  and  recorded  the  four  ratings  . 

Part  IV  of  the  testing  required  subjects  to  make  decisions  about  receiving  hits  from  enemy 
ships.  Subjects  were  shown  the  five  sets  of  individual  ship  pictures.  Each  set  contained  12 
pictures.  Each  picture  was  projected  for  seven  seconds.  Subjects  were  given  an  answer  sheet 
for  each  picture.  These  answer  sheets  showed  the  paths  with  the  dots  where  the  Battle  Group 
could  be  hit.  Subjects  were  to  circle  the  dot  (or  dots)  to  Irxiicate  where  each  shown  enemy  ship 
could  hit  the  subjects'  Battle  Group. 

Part  V  of  the  test  provided  subjects  with  diagrams  of  the  straight  and  curved  paths.  Subjects 
were  asked  to  shade  and  label  the  areas  where  enemy  ships  capable  of  scoring  hits  could  be 
located.  There  was  no  time  limit  for  this  task. 

4.3.  Results  snd  discussion 

Data  collected  support  the  hypothesis  that  the  subjects  developed  a  memory  aid  that 
facilitated  path  evaluation.  Furthermore,  the  data  are  consistent  with  that  aid  being  a  fuzzy 
schema  for  estimating  ship  hits.  The  data  do  not  support  the  use  of  contour  discriminators  that 


dMde  the  task  pictures  into  regions  of  zero,  one,  and  two  hits.  There  is  strong  evidence  that  the 
indicator/  counteiindicator  features  played  no  role  In  alternative  selection. 
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4.3.1.  Memory  In  task  performance 

The  subjects'  ability  to  correctly  rank  the  altematives  suggest  that  some  kind  of  memory  aid 
developed.  Subjects  are  able  to  rank  the  options  much  better  than  guessing  alone  would  allow. 
Were  they  only  guessing,  the  subjects  would  have  picked  the  right  alternative  on  only  eight  of  24 
pictures.  In  fact,  the  average  number  of  pictures  correctly  assessed  was  13.95  of  the  24 
presented.  This  number  is  significantly  better  than  the  number  right  that  would  be  predicted  from 
guessing,  t  (19)  -  10.635,  p  <  .01 .  Figure  4-3  shows  how  often  subjects  selected  the  correct 
alternative. 

A  more  complete  analysis  of  subjects'  ability  to  rank  options  considers  not  only  their  ability  to 
identify  the  best  option,  but  also  their  ability  to  correctly  rank  all  three  options.  Ranking  the  three 
options  requires  determining  the  better  option  for  each  of  the  three  possible  pairs:  straight  v. 
curved;  straight  v.  stay;  and  curved  v.  stay.  If  subjects  were  only  guessing,  then  their  answers  for 
each  pair  would  be  equally  probable  and  they  would,  on  the  average,  correctly  pick  the  better 
option  half  of  the  time,  thus  scoring  1 .5  out  of  3  possible  points  for  each  picture.  The  expected 
score  for  random  option  selection  for  the  24  pictures  would  be  36.  The  subjects'  actual  mean 
score  (X  3  51 .750 )  is  significantly  different  than  the  expected  score  for  guessing  (36);  t  (19)  > 
11.990,  p  <  .01. 

4.3.2.  Schemata  for  Individual  ships 

The  subjects  are  developing  data  in  memory  to  help  them  estimate  hits.  Figure  4-4  shows 
their  accuracy  for  estimating  hits  for  the  ships  seen  in  training.  Their  estimates  are  far  more 
accurate  than  chance.  Data  from  part  IV  also  indicates  that  subjects'  memory  helps  them  estimate 
ship  hits. 

In  part  IV  subjects  were  asked  to  identify  the  exact  set  of  path  hit  points  (the  drdes  on  the 
paths)  that  Individual  ships  can  hit.  Subjects  were  tested  on  60  ships  of  which  48  were  In 
locations  that  had  not  been  seen  in  training.  Subjects  performing  at  random  would  identify  the 
correct  set  very  infrequently.  Those  adopting  a  simple  rule  of  assuming  that  a  ship  would  always 
hit  only  the  ’nearest"  path  hit  point  would  get  approximately  21  correct  (or  35%).  Subjects  actually 
got  an  average  of  42.3  correct  (or  71%). 

The  data  in  memory  to  support  ship  hit  estimates  includes  more  than  just  the  number  of  hits 
from  ships  seen  hi  training.  It  Includes,  In  addition,  data  that  enable  them  to  estimate  hits  from 
ships  at  new  locations.  If  subjects  could  use  only  their  memory  of  the  number  of  hits  from 
previously  seen  ships,  then  they  would  estimate  the  hits  from  previously  seen  ships  better  than 
the  hits  from  ships  in  new  locations.  In  fact,  subjects  can  estimate  the  number  of  hits  from  ships  at 
some  new  locations  as  well  as  they  can  estimate  the  hits  from  previously  seen  ships.  Table  4-3 
summarizes  the  data  from  part  IV  of  the  test.  It  shows  that  subjects  do  equally  well  for  previously 
seen  ships,  for  new  ships  at  locations  which  are  mirror  reflections  of  the  previously  seen  ships, 
and  for  new  ships  that  are  the  same  distance  from  the  path  hit  points  as  a  previously  seen  sNp. 
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Figure  4-4:  Accuracy  of  subjects'  ship  hit  estimates  for  seen  ships? 


*  set  A,  part  IV,  Experiment  1 
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(OUT  FROM  A) 
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NO 

DISPLACEMENT 

ANGULAR 

DISPLACEMENT 

RADIAL 

DISPLACEMENT 

ANGULAR 

DISPLACEMENT 

RADIAL 

DISPLACAEMENT 

9.30 

9.85 

6.95 

9.05 

7.15 

Table  4-3:  Mean  score  for  number  of  correct  estimates  of  ship  hits  for  different 
categories  of  ships  (12  ships  per  category). 


ID 

c 

D 

UJ 

A 

t:  -1.078 

sig:  .295 

t:  6.092 

sig;  .000 

t:  0.567 

sig:  .577 

t;  6.750 

sig:  .000 

B 

— 

t:  5.659 

sig:  .000 

t;  1.752 

sig:  .096 

t;  5.156 

sig;  .000 

C 

— 

— 

t:  -4.228 

sig:  .000 

t:  -0.535 

sig:  .599 

D 

— 

— 

— 

t:  4.790 

sig;  .000 

Table  4-4:  t-statlstics  and  levels  of  significance  between  scores  for 
sets  A  through  E  using  paired  t-tests  with  dfs19  (part  IV,  Experiment  1). 
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There  is  no  significant  difference  between  ability  to  estimate  hits  from  ships  at  these  points. 

^  Subjects  are  significantly  worse,  however,  at  estimating  hits  from  ships  that  are  displaced  radially 

\  fll  frompreviously  seen  ships.  Table  4-4  summarizes  these  data. 


These  data  suggest  that  subjects'  ability  to  estimate  hits  from  ships  in  new  locations  is 
supported  by  schemata  of  the  kind  discussed  in  section  3.2.2  The  schema  hypothesis  proposes 
that  each  schema  contains  the  location  and  number  of  hits  from  a  previously  seen  ship  as  well  as 
feature  data  that  enable  subjects  to  evaluate  the  significance  of  ship  displacements  from  the 
location  of  the  previously  seen  ship.  With  these  features  they  are  able  to  estimate  the  number  of 
hits  from  ships  at  new  positions. 

Subjects  apparently  understood  the  symmetry  of  the  ship  hit  estimation  and  were  able  to 
use  this  understanding  to  estimate  the  hits  from  ships  at  new  locations.  They  understood  that 
angular  changes  in  a  ship's  location  with  respect  to  a  path  hit  point  did  not  change  that  ship's 
ability  to  score  a  hit  at  that  point.  Even  large  physical  displacements  made  no  difference,  so  long 
as  radial  distance  to  the  path  hit  point  did  not  change.  Quite  small  radial  displacements,  however, 
significantly  affected  their  ability  to  estimate  the  hits  from  the  displaced  ship.  A  fuzzy  schema  with 
these  two  features,  angular  and  radial  displacement,  was  described  in  section  3.2.2.  The  data  in 
Table  4-4  are  consistent  with  that  schema  model. 

Another  kind  of  memory  reference  structure  potentially  able  to  support  subjects'  ship  hit 
estimates  are  the  discriminator  contour  curves  that  separate  the  task  map  into  areeis  of  zero,  or^. 
and  two  hits.  These  contours  were  described  in  section  3.2.2.  If  subjects  had  such  contours  in 
their  memories,  then  they  could  estimate  the  number  of  hits  by  comparing  the  location  of  shown 
ships  to  the  locations  of  the  contours.The  data  collected  from  this  experiment  suggest  that 
subjects  do  not  use  such  contour  curves. 

In  part  V  of  the  experiment  subjects  were  given  a  paper  with  the  printed  Battle  Group  path 
alternatives.  They  were  asked  to  draw  on  this  paper  the  areas  from  which  ships  could  obtain  hits. 
Their  drawings  were  highly  inaccurate  in  defining  the  actual  areas  from  v4iich  ships  could  obtain 
hits.  Figure  B-1  in  appendix  B  is  an  example  of  a  typical  drawing.  These  drawings  show  that  most 
subjects  acquired  some  general  understanding  of  the  relationships  between  ship  locations  and 
ship  hits.  Fifteen  of  the  subjects'  drawings  showed  that  ships  further  along  the  path  could  score 
hits  from  initial  locations  farther  from  the  path.  Six  subjects  drew  circles,  which  is  the 
mathematically  correct  shape  of  the  contour.  Although  they  seemed  to  understand  some  of  the 
important  factors  affecting  the  ability  of  ships  to  score  hits,  subjects  estimated  very  poorly  the  size 
of  the  areas  from  which  sNps  could  score  hits.  Subjects'  ship  hit  estimates  in  part  IV  were 
consistent  with  their  drawings  in  part  V  only  53%  of  the  time  (Figure  4-5). 
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The  poor  quality  of  these  drawings  argues  against  subjects  using  such  contours  as  the 
basts  of  their  ship  hit  estimates,  but  does  not  preclude  such  use.  It  is  possible  that  people  could 
be  using  contours  for  ship  estimates  and  still  be  unable  to  draw  them  accurately. 
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FRACTION  OF  INDIVIDUAL  SHIP  HIT  ESTIMATES  (PART  IV) 
WHICH  CAN  BE  PREDICTED  FROM  DRAWING  (PART  V). 


Figure  4-5:  Inconsistency  between  ship  hit  estimates  given  and 
those  Inferred  from  drawing  (Experiment  1). 


4.3.3.  Indicator/counttr-IndIcators  for  altamative  aelectlon 


A  comparison  o1  subjects'  evaluations  of  paths  in  the  first  part  of  this  experiment  cieariy 
shows  that  they  did  not  use  the  indicator/  counterindicator  features  to  help  them  identify  the 
preferred  option.  The  indicator/  counterindicator  features  were  consistent  with  the  first  choice 
answer  in  the  first  12  test  pictures,  but  were  inconsistent  in  the  second  12  test  pictures.  If 
subjects  were  using  the  indicator/  counterindicator  features,  then  they  would  select  the  correct 
option  more  often  on  the  first  twelve  pictures  than  they  would  in  the  second  twelve.  They  did  not 
do  so.  Subjects  did  better  on  the  second  half  of  the  test  than  on  the  first  half.  The  number  right 
on  the  first  half  of  the  test(  X  >  6. 1 5 )  is  significantly  worse  than  the  number  right  on  the  second 
half  of  the  test  ( X  -  7.80);  t  (1 9)  -  -3.885,  p  <  .001 .  Figure  4-6  shows  these  data. 

Why  did  subjects  do  better  on  the  second  half  of  the  test?  Two  explanations  are  offered. 

(1 )  Subjects  may  have  become  better  at  pacing  themselves  in  the  timed  test  situation  and, 
therefore,  tended  to  do  better  on  later  pictures.  It  is  possible  that  this  would  have  made  a  small 
difference  in  performance,  but  it  is  unlikely  that  it  could  account  for  this  significant  dlfferer>ce.  (2) 
The  pictures  on  the  first  half  of  the  test  may  have  been  more  difficult  for  the  subjects  than  those 
on  the  second  half  of  the  test.  This  was  explored  further. 

The  materials  were  designed  with  the  intent  that  both  sets  of  pictures  would  be  equally 
difficult.  The  same  ships  were  shown  the  same  number  of  times  within  each  set  and  the 
differences  between  the  options  were  equivalent  between  the  sets.  A  closer  examination  of  the 
materiais,  however,  revealed  that  the  path  choices  in  the  second  set  could  be  nwre  difficult  for 
subjects  to  rank. 

Hits  from  some  ship  locations  were  generally  less  accurately  estimated  than  hits  from  other 
locations.  Some  '2'  hit  ships  are  nearer  to  the  '1'  hit  area  than  others.  Some  'V  hit  ships  are  nearer 
to  the  '2  hit  area  and  some  are  nearer  to  the  'O'  hit  area  Ship  ambiguity  scores  that  reflect  these 
estimation  differences  were  computed  for  each  of  the  24  ships  in  the  test  pictures  using  the  ship 
hit  data  from  part  IV.  For  example,  a  ship  that  was  accurately  rated  '2'  by  75%  of  the  subjects  would 
have  a  ship  ambiguity  score  of  2.0  -  .25.  Path  ambiguity  scores  were  calculated  from  the  ship 
ambiguity  scores  by  summing  the  upper  and  lower  estimates  of  Nts  from  each  ship  along  each 
path.  A  comparison  of  the  path  ambiguity  scores  (for  the  straight  path,  the  curved  path,  and  the  '6' 
hit  cut  off  for  the  stay  option)  in  the  two  sets  indicates  that  the  first  set  of  pictures  are  somewhat 
more  difficult.  There  are  7  pictures  in  the  first  twelve  in  which  the  upper  and  lower  bournfs  of  ship 
hits  alorrg  the  path  either  overfap  each  other  or  overlap  the  stay  option  criteria  of  six  hits.  In  the 
second  twelve  there  are  only  5  pictures  having  an  ambiguity  overlap.  In  addition  the  average  size 
of  the  ambiguity  overlap  is  larger  in  the  first  set.  Thus,  the  first  12  pictures  would  be  perceived  as 
more  difficult  than  the  second  twelve  pictures  . 

In  spite  of  these  differences  in  the  perception  of  picture  difficulty,  it  is  dear  that  subjects 
were  not  using  the  indicator/  counterindicator  features  In  making  their  dedsions.  Furthennore, 
when  debriefed,  they  indicated  that  they  were  not  consciously  aware  of  these  cues. 
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The  second  experiment  was  designed  to  study  more  closely  issues  raised  by  the  previous 
experiment.  The  decision  task  was  described  In  section  3.3.  Figure  3-5  is  an  example  of  the 
situation  presented  to  subjects.  As  in  the  first  experiment,  subjects  were  asked  to  select  the  best 
option  in  response  to  this  barrier;  to  traverse  it  along  the  straight  path;  to  traverse  it  along  the 
curved  path,  or  to  not  traverse  it,  staying  in  front  of  the  barrier. 

5.1.  Issues  and  design  overview 

Data  from  the  first  experiment  suggest  that  people  use  schema-like  memory  structures  for 
estimating  ship  hits,  and  clearly  indicate  that  they  do  not  use  indicator/  counterindicator  features 
for  selecting  alternatives  without  evaluating  expected  hits  along  the  path.  They  also  suggest  that 
people  do  not  base  their  hit  estimates  on  contours  separating  the  situation  diagram  into  areas  of 
zero,  one,  and  two  ship  hits.  Because  the  task  in  the  first  experiment  was  so  complex  and 
because  the  path  quality  depended  on  submarine  patrol  areas  eis  well  as  on  ship  hits,  H  was  not 
possible  to  investigate  the  connection  between  ship  hit  estimates  and  alternative  selection.  In 
addition,  the  task  complexity  may  have  inhibited  subjects  from  forming  ’wholistic'  path  schemata 
for  path  evaluation. 

In  order  to  examine  the  connection  between  estimates  of  ship  hits  and  the  selection  of  a 
decision  alternative  and  in  order  to  encourage  the  formation  of  wholistic  path  schema,  ERA 
simplified  the  previous  task  for  this  experiment.  The  barriers  in  this  experiment  contain  only  ships. 
There  are  no  submarines.  Ships  can  attack  anywhere  along  the  paths  and  do  not  need  to  change 
course  to  score  a  second  hit.  In  addition,  in  order  to  test  how  the  type  of  memory  staicture  formed 
might  be  influenced  by  training.  ERA  trained  subjects  in  three  different  ways. 

Table  5-1  summarizes  the  test  procedures  designed  to  address  the  these  issues. 

5.2.  Methods 
5.2.1.  Subjects 

The  subjects  were  undergraduate  students  at  Qeorge  Mason  University  who  received 
either  course  credit  or  payment  for  their  participation.  Subjects  were  run  in  small  groups  of 
approximately  four  each.  Each  subject  was  assigned  to  one  of  three  different  training  condfilons; 
"measure",  "curve",  and  "outcome".  Eighty-three  subjects  participated  in  the  study,  28  in  the 
measure  group;  23  in  the  curve  group;  and  32  in  the  outcome  group.  Twenty  subjects  reached 
criterion  for  each  type  of  training. 
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ISSUE 


EXPERIMENT  PROCEDURE 


1 .  Subjects  trained  to  calculate  1 . 

outcomes  and  use  rules  will 

develop  memory  reference  data  to 
help  them  rank  path  alternatives. 

2.  Subjects' memories  allow  them  to  2. 

accurately  estimate  ship  hits. 

3.  Subjects  use  schemata  (similar  to  3. 

the  type  seen  in  task  1 )  for 
estimating  hits  from  individual 

ships. 

4 .  Subjects  abstract  a  mental  nrtap  or  4 . 

set  of  contours  that  they  use  in 
metking  their  decisions. 


5.  Subjects  select  the  decision  5. 

alternative  by  an  "eyeball  and 

count"  process,  in  which  subjects 
quickly  estimate  and  sum 
individual  ship  hits. 

6.  Subjects  use  a  wholistic  path  6. 

schema  to  evaluate  and  estimate 

path  quality. 


7.  Training  mode  does  not  affect  7. 
subject  performance  because  all 
subjects  are  trained  to  criterion. 


Subjects  are  allowed  very  little  time  in  which  to 
estimate  the  number  of  hits  from  each  ship  and 
to  rank  the  options.  The  accuracy  and 
consistency  of  these  rankings  is  computed. 

The  accuracy  and  consistency  of  subjects' 
ship  hit  estimates  are  computed. 

Subjects  estimate  hits  from  ships  in  several 
locations.  Consistency  of  subject  responses 
with  schema  predictions  is  evaluated. 


Subjects  outline  areas  from  where  ships  can 
score  hits.  The  size  and  shape  of  these 
contours  are  examined.  In  addition  ship  hit 
estimates  predicted  from  the  drawn  contour 
curves  are  compared  with  sNp  hK  estimates 
actually  given  in  parts  IV  and  VII  to  determir^e 
whether  they  are  significantly  different 

Son>e  subjects  are  shown  the  ground  truth 
contours  during  training.  The  accuracy  of  the 
ship  estimates  by  this  group  is  compared  with 
the  accuracy  by  other  groups. 

Subjects  estimate  hits  from  individual  ships. 
These  estimates  are  used  to  predict  their  path 
rankings.  The  consistency  of  these  predicted 
rankings  with  their  actual  rankings  in  parts  I  and 
VI  is  computed. 

Subjects  provide  ratings  of  path  quality. 

These  ratings  are  used  to  predict  their  path 
rankings.  The  consistency  of  these  predicted 
rankings  with  their  actual  rankings  in  parts  I  and 
VI  is  computed. 

Data  are  analyzed  separately  for  each  training 
group.  Group  differences  reflect  the  influence 
of  tr^ning  on  performance. 


Tabla  5-1.  Issues  arid  experimental  procedures  (Experiment  2). 


5.2.2.  Materials 

Materials  consisted  of  three  sets  of  12  pictures  each  (initial  training  pictures,  training  to 
criteria  pictures,  and  test  pictures)  and  30  single  ship  pictures.  The  two  training  sets  were 
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packaged  into  training  booklets  with  prepared  answer  sheets  for  each  picture  and  a  sheet  on 
which  subjects  could  record  their  answers.  The  test  set  and  single  ship  pictures  were  made  into 
siides  for  projection  with  a  siide  projector.  A  test  booklet  contained  directions  and  pages  on  which 
subjects  couid  record  their  responses.  The  pictures  were  identical  for  all  groups,  but  the  method 
for  calculating  the  ship  hits  varied. 

All  pictures  had  the  same  thick  lines  indicating  the  straight  and  curved  path  options.  The 
number  of  ships  in  the  pictures  (training  and  test)  varied  from  5  to  10.  The  sets  of  pictures  (initial 
training,  training  to  aiteria,  and  test)  were  matched  as  dosely  as  possible.  Each  set  had  the  same 
ships  shown  the  same  number  of  times.  Each  set  had  the  same  pairs  of  path  scores.  The 
individual  pictures  differed  only  in  the  number  and  location  of  ships.  No  two  pictures  were  alike. 
Every  picture  differed  from  the  others  by  at  least  two  ships. 

The  training  pictures'  answer  sheets  and  supplemental  materials  were  different  for  different 
training  groups.  During  the  initial  training  (pictures  1-12),  the  measure  group  had  rulers  that  were 
used  to  measure  and  mark  the  ship  movements.  The  measure  group’s  answer  sheets  for  both 
the  initial  training  and  the  training  to  criteria  showed  the  lines  of  measured  ship  movements.  For 
the  curve  group  the  training  pictures  and  answer  sheets  for  both  training  sets  contained  the 
ground  truth  contour  curves  that  separated  the  situation  map  into  areas  In  which  ships  could  score 
zero,  one,  or  two  hits.  The  outcome  group's  answer  sheets  for  initial  training  and  training  to  criteria 
marked  the  number  of  hits  that  ship  could  make  next  to  each  ship. 

5.2.3.  Procedure 

Training  procedure 

There  were  three  versions  of  training:  measure,  curve,  and  outcome.  In  the  measure 
training,  subjects  were  given  rulers  that  showed  how  far  a  hostile  ship  can  move  in  one  time  unit. 
Subjects  were  trained  to  measure  the  ship  movements  in  order  to  determine  ship  hits  for  each 
path.  In  the  curve  group,  the  pictures  contained  the  ground  truth  contour  curves  .  Subjects 
could  use  these  curves  to  determine  the  number  of  hits  each  ship  could  get  for  each  path.  In  the 
outcome  group,  subjects  were  told  to  guess  how  many  hits  each  hostile  ship  could  score  on  each 
path.  They  then  compared  their  guesses  with  the  correct  answers  on  answer  sheets.  In  all  cases, 
subjects  were  told  to  compute  the  total  number  of  hits  for  each  path  and  to  then  make  a  decision 
using  the  mle:  "Chose  the  path  with  the  least  number  of  potential  hits  if  that  number  is  four  or 
less.  If  that  number  is  more  than  four,  stay  where  you  are." 

After  the  initial  training,  subjects  were  provided  with  a  second  set  of  pictures  and  trained  to 
criterion.  These  pictures  were  the  same  for  all  groups,  but  were  followed  by  answer  sheets  that 
varied  according  to  the  training  mode.  Subjects  recorded  their  rankings  of  the  options  on  a  score 
sheet  and  then  compared  their  first  choice  with  the  correct  first  choice.  The  subjects  scored  their 
own  papers.  Subjects  who  got  nine  or  more  correct  passed  the  training  to  criterion.  Data  from 
subjects  who  did  not  pass  criterion  was  not  analyzed. 

The  test  portion  of  the  experiment  had  seven  parts.  Pictures  in  the  test  set  were 
randomized  each  time  the  set  was  shown.  In  part  I  subjects  ranked  the  "straight,"  "curved,"  and 


'stay'  options  for  each  picture  in  the  test  set.  Each  of  these  pictures  were  projected  for  10 
seconds.  Subjects  had  an  additional  1 0  seconds  to  record  their  rankings  (straight  path,  curved 
path,  or  stay). 

In  part  II  subjects  rated  each  path  on  a  scale  of  1  to  10  with  respect  to  how  good  the  path 
was  at  blocking  the  Battle  Group.  The  set  of  test  pictures  was  presented  twice  In  part  II.  Each 
picture  was  projected  for  5  seconds.  In  the  first  presentation  (part  IIA),  subjects  rated  the  straight 
path.  In  the  second  presentation(part  I  IB),  subjects  rated  the  curved  path. 

In  part  III  subjects  rated  each  of  the  paths  on  a  scale  of  1  to  10  with  respect  to  a  second 

statement;  "Many  ships  near  the  ( _ )  path.'  Again,  each  set  was  presented  twice  with  each 

picture  shown  for  5  seconds.  In  part  IIIA,  subjects  rated  the  straight  paths  and,  in  part  IIIB,  subjects 
rated  the  curved  paths. 

In  part  IV  the  single  ship  pictures  were  projected  for  8  seconds  each.  Subjects  entered  on 
the  answer  sheet  their  confidence  that  the  shown  ship  could  score  'O’,  '1',  or  '2'  htts. 

In  part  V  subjects  were  provided  with  paper  showing  only  the  curved  and  straight  paths,  and 
were  asked  to  draw  contours  separating  the  areas  from  which  ships  could  score  zero,  one,  or  two 
hits.  There  was  no  time  limit  for  this  part. 

Part  VI  repeated  part  I  (ranking  the  options)  and  part  VII  repeated  part  IV  (recording  the  sNp 
hits  for  the  single  ships). 

5.3.  Results  and  discussion 
5.3.1.  Memory  In  task  performance 

Subjects  have  developed  data  in  memory  that  allow  them  to  do  this  task.  Subjects  are  able 
to  rank  the  options  much  better  than  guessing  alone  would  allow.  The  12  test  pictures  were 
shown  in  parts  I  and  VI  (a  total  of  24  items).  Figure  5-1 A  shows  the  distribution  of  the  numbers 
correct  for  each  of  the  three  training  groups.  An  item  was  considered  connect  if  the  option  ranked 
first  choice  was  the  best  option.  The  means  for  number  correct  are  1 7.00  (measure), 

17.60(curve),  and  1 6. 1 5(outcome)  which  are  all  significantly  different  from  the  number  correct 
predicted  by  guessing  (8),  t  (19)  -  12.728,  15.884,  15.249,  respectively,  p  <  .01.  The  scoring 
method  used  in  the  first  experiment  was  again  used  to  meaisure  how  well  subjects  could  rank  the 
three  options.  There  were  3  points  possible  for  each  picture  for  a  total  of  72  points  possible. 
Figure  5-1 B  shows  the  distribution  of  these  scores  for  each  training  group.  The  means  for  the 
scores:  59.45(measure),  61.50(curve),  and  57.40(outcome),  are  also  significantly  different  from 
the  score  expected  were  subjects  guessing  at  random  (36),  t(19)  -  19.301,  24.227,  20.942, 
respectively,  p  <  .001. 

T raining  mode  generally  did  not  affect  task  performance.  The  differences  in  the  number 
correct  or  in  the  ranking  scores  between  the  measured  and  the  outcome  groups  are  not 
significant,  t(38)  -  .959(p  ■  .344),  1.291(p  ■  .204),  respectively.  Similarly,  the  differences 
between  the  curve  and  measure  groups  are  insignificant,  t(38)  -  .645(p  ■  .523),  1.275(p  »  .210), 


22 


24 


2  4  6  8  10  12  14  16  18  20 

NUMBER  OF  TIMES  WHEN  BEST  OPTION  WAS  RANKED  #1(OUT  OF  24) 


Figure  5-1  A:  Accuracy  of  subjects'  decisions:  the  number  correct  (Experiment  2). 
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RANKING  SCORES  (OUT  OF  72) 


Figure  5-1 B:  Accuracy  of  subjects’  decisions:  ranking  scores  (Experiment  2) 
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respectively.  For  the  curve  and  outcome  groups  the  dHlerence  between  the  number  correct  is 
not  significant.  t(38)  -  1 .797(p  -  .08).  but  the  difference  between  their  ranking  scores  is 
significant.  t(38)  =  2.795.  p  ■  .008. 

The  test-retest  consistency  of  subjects’  rankings  also  supports  the  existence  of  memory 
reference  structures.  Figure  5-2  shows  the  consistency  of  rankings  made  in  parts  I  and  VI  using 
the  scoring  method  previously  described  (section  4.3.1). 

Clearly,  subjects  can  identity  preferred  options  by  lookirrg  briefly  at  the  configuration  of 
hostile  ships.  Since  they  do  not  have  the  tools  to  measure  ship  hits  nor  the  time  to  cewefully 
compute  hits  along  paths,  ttiey  must  be  using  some  other  process  to  perform  this  task, 
presumably  one  based  on  memory. 

5.3.2.  Memory  for  ships 

This  process  may  employ  memory  aided  ship  hit  estimates,  for  during  training  subjects 
acquired  an  ability  to  recognize  previously  seen  ships  and  to  remember  the  hits  assodiated  with 
them.  This  ability  can  be  measured  both  in  terms  of  the  consistency  of  subjects'  estimates,  arKl 
also  in  in  terms  of  their  accuracy.  Consistency  is  measured  by  the  correlation  between  subjects 
test-retest  responses.  Accuracy  is  measured  by  how  doseiy  their  estimates  match  the  ground 
truth  answers. 

Subjects  are  very  consistent  in  their  estimates  of  shio  hits.  The  coefficient  of  correlation 
between  test-retest  ship  hit  estimates  in  parts  IV  and  VII  was  greater  than  .700  for  most  subjects. 
Very  few  had  correlations  below  .500.  Figure  5-3  shows  the  distribution  of  these  correlation 
coefficients. 

Subjects  are  also  accurate  in  their  ship  estimates.  Figure  5-4  show  the  mean  estimate  of 
hits  for  each  ship  shown. 

5.3.3.  Schemata  for  Individual  ships 

Schemata  for  individual  ships  enable  subjects  to  judge  the  effect  of  changes  in  ship 
location  on  hits  from  that  ship.  In  this  task,  the  schema  data  would  indicate  how  the  number  of  hits 
for  a  ship  changes  as  its  location  changes  either  in  a  direction  parallel  to  or  perperKflcular  to  the 
two  ship  paths. 


These  schema  are  reflected  by  the  differences  in  subjects'  ship  hit  estimates  between  pairs 
of  ships  that  are  either  the  same  distance  along  the  path  but  at  different  distances  from  the  path, 
or  are  at  the  same  distance  from  the  path  but  are  at  different  distances  along  the  path.  For 
example,  ships  #4  and  #5  can  both  score  2  hits  arKl  are  the  same  distance  from  the  straight  path, 
but  ship  #4  is  further  along  the  path  than  ship  #5  and,  therefore,  has  more  time  to  reach  the  path. 
Subjects  are  generally  more  confident  that  ship  #4  is  capable  of  scoring  2  hits  than  they  are  that 
ship  #5  is  capable  of  scoring  2  hits  (12  subjects  were  more  confident  of  ship  #4,  47  subjects  were 
equally  confident,  and  1  subject  was  more  confident  of  ship  #5).  The  same  relationship  exists 
between  ships  #8  and  #9  which  are  each  capable  of  scoring  1  hit.  Again,  sut^ects  were  gerrerally 
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Figur*  5-2:  Consistency  of  subjects'  decisions:  rsnkings 
(  parts  I  and  VI  -  Experiment  2). 


COEFFICIENTS  OF  CORRELATION  BETWEEN 
SHIP  HIT  ESTIMATES  (PARTS  IV  AND  VII) 


Figure  5-3:  Consistency  of  subjects'  ship  hit  estimates 
for  seen  ships  (Experiment  2). 


two  hits 


Accuracy  of  subjects'  ship  hit  estimates  (Experiment  2) 


more  confident  of  ship  #8  than  of  ship  #9  (  30  subjects  were  more  confident  of  ship  #8, 1 1 
subjects  were  equaiiy  confident,  and  19  subjects  were  nrxjre  confident  of  ship  #9). 

These  findings  support  the  schema  structure  for  individuai  ships  that  was  described  in 
chapters  3  and  4. 

5.3.4.  Evidence  against  contour  model 

The  contour  model  proposes  that  subjects  have  in  memory  reference  curves  that  overlay 
the  situation  map,  dividing  it  into  areas  from  which  ships  can  get  no  hits,  one  hit,  or  two  hits.  They 
use  these  contours  to  estimate  hits  from  each  ship  by  comparing  the  location  of  each  ship  with 
these  contours.  Ships  outside  the  '0-r  contour  would  score  no  hits;  those  between  the'0-1' 
contour  and  the  "1-2"  contour  would  score  one  hits;  and  those  between  the  path  and  the  '1-2' 
contour  would  score  two  hits.  Ground  truth  contours  are  shown  in  Figures  3-7  and  5-4. 

This  experiment  provides  two  kinds  of  evidence  against  the  existence  or  use  of  such 
contours:  the  poor  quality  of  contours  drawn  by  the  subjects,  and  the  lack  of  impact  that  exposure 
to  ground  truth  contours  has  on  the  accuracy  of  subject's  ship  hit  estimates. 

If  subjects  have  these  contours  in  their  memories,  then  they  may  be  able  to  draw  them.  If 
subjects  are  estimating  ship  hits  by  comparing  the  location  of  ships  with  the  positions  of  the 
contours,  then  the  ship  hit  estimates  predicted  from  the  contour  locations  should  be  consistent 
with  the  hit  estimates  provided  directly  by  the  subjects. 

Figure  B-2  in  Appendix  B  shows  a  sample  drawing.  In  general,  the  drawings  show  that 
subjects  understand  that  the  areas  from  which  ships  could  score  hits  get  wider  further  along  the 
path  and  that  the  lines  defining  the  '2'  and  T  hit  areas  are  parailei.  Table  5-2  shows  the  number  of 
drawings  in  each  group  of  20  subjects  that  were  consistent  with  these  ideas.  The  very  high 
performance  of  the  curve  group  is  not  surprising,  for  these  subjects  saw  the  ground  truth 
contours  during  training 
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Subjects  are  not  basing  their  ship  hit  estimates  on  the  contours  that  they  draw.  The 
estimates  that  wouid  be  predicted  from  the  drawn  curves  were  obtained  by  superimposing  tt)e  30 
ship  iocations  on  each  subject's  drawing  to  obtain  ship  hit  scores  for  each  ship.  These  inferred 
estimates  were  then  compared  with  the  estimates  provided  directly  by  the  subjects.  For  each 
subject  three  paired  t-tests  were  done:  (1 )  between  ship  estimates  given  in  parts  IV  and  VII;  (2) 
between  ship  estimates  of  part  IV  and  the  drawing-derived  estimates;  and  (3)  between  ship 
estimates  on  part  VII  and  the  drawing-derived  estimates.  In  most  cases,  there  were  no  significant 
differences  (p  <  .05)  between  the  test-retest  ship  estimates  given  in  parts  IV  and  VII.  but  there 
were  significant  differences  between  the  drawing-predicted  estimates  and  each  of  the  other 
estimates. 

Figure  5-5  shows  the  number  of  cases  in  which  the  differences  were  significant  for  each 
training  group.  As  illustrated,  the  drawings  of  the  curve  group,  the  group  that  saw  the  ground 
truth  curves  during  training,  were  more  consistent  with  their  ship  hit  estimates  than  were  either  of 
the  other  group's  drawings.  However,  even  for  the  curve  group,  the  consistency  between  the 
hits  predicted  from  the  drawings  and  the  test  or  retest  estimates  was  much  less  than  the 
consistency  between  the  test-retest  estimates.  It  seems  clear  that  subjects  are  not  using  the 
contours  that  they  drew  to  estimate  ship  hits. 

There  is  a  possibility,  however,  that  subjects  have  such  contours  in  memory  and  are  using 
them  to  estimate  ship  hits,  but  are  unable  to  draw  ttiem  accurately.  We  assume,  for  example,  that 
people  recognize  their  friends  by  comparing  the  faces  of  people  they  see  with  memory  reference 
data  desaibing  the  faces  of  their  friends.  Most  people  cannot  draw  these  faces  from  memory  very 
well.  Their  inability  to  do  so  does  not  preclude  their  having  memory  data  for  faces. 

This  difficvlty  was  addressed  by  comparing  the  ship  hit  estimation  ability  of  subjects  trained 
on  ground  truth  contours  with  those  not  trained  on  these  contours.  If  subjects  are  using  these 
contours  to  estimate  ship  hits,  then  subjects  with  more  accurate  contours  would  estimate  ship  hits 
more  accurately  than  those  with  less  accurate  contours.  Subjects  in  the  ’measure’  or  ’outcome’ 
training  groups  would  need  to  abstract  these  contours  from  their  measurements  or  observed 
outcomes.  Those  in  the  ’curve’  group  saw  #ie  accurate  contours  during  training.  Therefore,  it  is 
likely  that  those  in  the  measure  and  outcon^e  groups  would  have  less  accurate  contours  than 
those  in  the  curve  group.  If  so.  and  if  they  are  using  these  contours  to  estimate  ship  hits,  then 
subjects  in  the  measure  and  outcome  groups  should  estimate  ship  hits  less  accurately  than  those 
in  the  contour  group. 

While  the  curve  groups'  drawings  are  indeed  better,  their  ship  hit  estimates  are  not.  Table 
5-3  shows  the  average  error  in  ship  hit  estimates  for  each  group.  The  differences  between  the 
groups  were  not  significant  using  unpaired  t-tests  (t  -  0.1 12,  sig  -  .91 1  between  the  measure  and 
curve  groups;  t  ■  -0.576,  sig  -  .567  between  the  measure  and  outcome  groups:  and  t  -  -0.457, 
sig  -  .649  between  the  curve  and  outcome  groups.) 
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BETWEEN  ESTIMATES  PART  IV  AND  ESTIMATES  PART  VII 

BETWEEN  ESTIMATES  PART  IV  AND  PREDICTIONS  FROM  DRAWINGS 
BETWEEN  ESTIMATES  PART  VII  AND  PREDICTIONS  FROM  DRAWINGS 


Figure  5*5:  Inconsistency  between  different  ship  hit  estimates 
using  paired  Meats  (Experiment  1). 
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Measure  group 

Curve  group 

Outcome  group 

0.313 

0.306 

0.282 

Tabis  5-3:  Mean  error  in  ahip  hit  eatimatlon. 

5.3.5.  Path  evaluation  from  ahIp  hita 

Since  subjects  are  able  to  estimate  ship  hits  using  schemata,  they  could  be  using  these 
estimates  to  select  arrKing  the  straight,  curve,  and  stay  options.  During  testing  subjects  did  not 
have  the  tools  to  calculate  the  potential  hits,  nor  did  they  have  the  time  to  carefully  estimate  the 
potential  hits.  Yet,  rather  than  abandon  the  calculations  and  projections  learned  in  training, 
subjects  reported  in  debriefings  that  they  tried  to  make  their  decisions  by  quickly  summing  the 
ship  hit  estimates  along  each  of  the  paths.  Many  subjects  reported  using  this  ’eyeball  and  count* 
strategy. 

The  data  are  consistent  with  subjects  using  this  strategy.  Using  the  average  of  each 
subject's  ship  hit  estimates  from  parts  IV  eind  VII,  option  rankings  were  predicted  by  summing  over 
the  hits  from  ships  in  each  of  the  pictures.  These  predicted  rauikings  were  then  compared  with 
each  of  the  subject's  actual  rankings  (parts  I  and  VI).  The  previously  described  scoring  system  was 
used  to  measure  consistency  between  rankings.  This  system  counted  the  number  of  straight  v. 
curved,  straight  v.  stay,  and  curved  v.  stay  pairings  that  had  the  same  order  in  the  predicted  and 
actual  rankings.  There  were  3  points  possible  per  picture.  In  addition,  in  order  to  provide  a 
baseline,  the  consistency  scores  between  the  test  and  retest  rankings  were  computed.  The 
distribution  of  these  consistency  scores  for  each  of  the  training  group  are  shown  in  Figures  5-6, 
5-7  and  5-8.  The  consistency  of  the  predicted  rankings  with  the  actual  test  or  retest  rankings  is 
very  similar  to  the  consistency  between  the  test-retest  rankings  themselves.  There  were  no 
significant  differences  (p  <  .05)  between  these  three  distributions  for  any  group  using  paired 
t-tests.  These  data  thus  support  the  hypothesis  that  subjects  were  indeed  basing  their  decisions 
on  the  quickly  estimated  ship  hit  sums. 

There  are  many  simple  summing  rules  that  subjects  may  have  been  using  when  evaluating 
the  straight,  curve,  and  stay  options.  WNIe  it  is  impossible  to  imagine  all  possible  rules,  it  is 
possible  to  bound  the  complexity  of  the  rule.  For  example,  the  following  rule,  which  does  not 
discriminate  between  ships  able  to  score  one  or  two  hits,  cannot  account  for  subjects' 
performance.  The  rule  is: 

Chose  the  path  with  the  least  number  of  ships  near  it,  providing  that  the  number  is 
less  than  five.  To  determine  If  a  ship  is  ’near”  a  path,  draw  a  line  from  the  starting 
position  to  the  midpoint  between  the  ends  of  the  two  options.  Ships  to  the  left  of  the 
line  are  "near"  the  straight  path.  Those  to  the  right  of  the  line  are  "near"  the  curved 
path. 
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Figure  5'6:  Consistency  of  decisions  for  the  measure  group: 

(A)  declslons(part  I)  with  declslons(part  VI); 

(B)  declslons(part  I)  with  predicted  decisions  based 
on  ship  estimates;  and 

(C)  declslons(part  VI)  with  predicted  decisions  based 
on  shlD  estimates. 
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Figure  5-7:  Consistency  of  decisions  for  the  curve  group: 

(A)  decislons(part  I)  with  decislons(part  VI); 

(B)  decisionsjpart  I)  with  predicted  decisions  based 
on  ship  estimates;  and 

(C)  declslons(part  VI)  with  predicted  decisions  based 
on  ship  estimates. 
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Figure  5-8:  Consistency  of  decisions  for  the  outcome  group: 

(A)  decisions(part  1)  with  declslons(part  VI); 

(B)  decisions(part  I)  with  predicted  decisions  based 
on  ship  estimates;  and 

(C)  declslons(Part  VI)  with  predicted  decisions  based 
on  ship  estimates. 
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Using  this  method,  the  expected  score  would  be  48  and  the  expected  number  correct 
would  be  1 1 .  The  actual  performance  of  subjects  was  better  than  what  would  be  expected  using 
this  rule,  t(19)  *  9.424  (score),  8.485  (number  correct)  for  the  measured  group;  t  -  12.826 
(score),  10.920  (number  correct)  for  the  curve  group;  and  t  «  9.199  (score),  9.636  (number 
correct)  for  the  outcome  group,  p  <  .001  for  all  groups.  Even  if  subjects  knew  how  to  define 
"near"  in  a  way  that  let  them  more  accurately  determine  with  which  path(s)  the  ships  should  be 
associated,  their  expected  score  would  be  53  and  their  expected  number  right  would  be  6  and 
both  of  these  are  significantly  (p  <  .001 )  worse  than  actual  subject  performance.  There  doesn’t 
seem  to  be  any  way  of  counting  ships  that  allows  subjects  to  perform  as  accurately  as  they  did. 
Subjects  miist  use  their  individual  ship  schema  to  estimate  actual  number  of  hits  per  ship  in  order 
to  rank  the  options  as  accurately  as  they  did. 

The  subjects'  ability  to  estimate  ship  hits  was  sufficiently  good  so  that  in  theory  they  could 
have  ranked  the  options  much  more  accurately  than  they  did.  The  consistency  scores  of  the 
rankings  predicted  from  subject's  ship  hits  with  the  ground  truth  rankings  was  63.1  for  the 
measure  group,  64.6  for  the  curve  group,  and  63.2  for  the  outcome  group  (72  would  be  a  perfect 
score),  whereas  the  consistency  scores  of  the  actual  rankings  with  ground  truth  was  59.45,  61.5, 
and  57.4  respectively  for  these  groups.  Time  pressure  is  not  likely  to  be  the  reason  why  subjects' 
actual  rankings  were  less  accurate  than  this  theoretical  maximum,  for  in  an  additional  experiment 
five  subjects  given  unlimited  time  to  rank  options  showed  the  same  pattern.  Ship  hit  and 
summing  variability,  inherent  in  each  estimation,  is  the  most  likely  explanation.  Such  variability 
must  be  proposed  to  account  for  the  consistency  scores  of  57.4,  60.3,  and  57.7  between  the 
test  and  retest  rankings.  Because  similar  consistency  scores  were  observed  between  the 
rankings  predicted  from  ship  hit  estimates  and  actual  rankings,  inherent  estimation  variability 
seems  able  to  account  for  subjects  failure  to  rank  options  as  accurately  as  the  theoretical  accuracy 
imposed  by  their  ship  hit  estimation  accuracy. 

5.3.6.  Evidence  against  path  schemata 

It  was  hypothesized  that  with  experience,  subjects  would  develop  "wholistic"  schemata. 
These  schemata  would  enable  subjects  to  recognize  an  entire  pattern  of  ships  along  a  path,  and 
to  associate  this  pattern  with  a  previously  computed  total  number  of  hits.  Subjects  with  such 
wholistic  schemata  would  not  need  to  "eyeball  and  count’  Rather  they  would  directly  associate  a 
pattern  with  a  sum  of  hits.  Wholistic  schemata  were  discussed  in  section  3.3.4. 

Although  the  "eyeball  and  count"  model  can  account  for  subjects'  rankings  of  options,  it  is 
possibie  that  subjects  also  employed  wholistic  schemata  when  ranking  options.  Using  wholistic 
schemata  to  rank  options  would  likely  entail  less  mental  effort  than  the  eyeball  and  count  strategy. 
If  subjects  had  such  schemata,  then  they  would  likely  prefer  to  use  them,  rather  than  eyeball  and 
count,  to  rank  options.  Further,  such  wholistic  schemata  would  also  support  subjects'  quantitative 
assessnf)ents  of  path  quality. 


Subjects'  qualitative  path  assessments  provide  some  evidence  that  they  did  not  use  such 
schemata  In  parts  II  and  III  of  the  experiment,  subjects  were  required  to  rate  the  paths.  In  part  II 
they  rated  paths  according  to  "how  good  the  path  was  at  blocking  the  Battle  Group."  In  part  III  of 
the  experiment,  subjects  were  required  to  rate  how  consistent  each  statement  "Many  ships  are 
near  the  straight(curved)  path."  is  with  each  path  in  each  picture.  If  subjects  abstracted  wholistic 
schemata  and  were  using  them  to  assess  path  quality  and  also  to  rank  options,  then  the  ratings 
given  in  part  II  and  III  should  be  consistent  with  and  correlate  with  the  decisions  they  made  in  parts  I 
arxl  VI. 

Although  it  is  impossible  to  predict  the  order  of  the  stay  option  with  respect  to  the  straight 
and  curved  options  from  these  data,  it  is  possible  to  predict  the  order  of  the  straight  and  curve 
options  (except  when  the  curves  are  given  the  same  ratings).  For  each  subject,  the  order  of  the 
rankings  for  the  straight  and  curve  option  was  compared  with  the  order  predicted  by  the  qualitative 
path  rankings  of  part  II  and  with  the  order  predicted  by  the  ship  hit  estimates.  Table  5-4 
summarizes  the  number  of  times  these  two  different  types  of  predictions  were  consistent  with  the 
actual  path  rankings.  When  the  order  of  the  rankings  predicted  by  ship  estimates  differs  from  the 
order  predicted  by  path  assessments,  then  decisions  are  more  consistent  with  ship  estimates 
than  with  path  assessments  made  in  part  II. 


training  group 

ranking 
predicted  by 
ship  estimate 
only 

ranking 
predicted  by 
path  assessment 
only 

total  number 
of  cases 

measure  group 

91 

13 

104 

curve  group 

40 

20 

60 

outcome  group 

69 

37 

106 

Table  5-4:  Comparison  of  decision  consistency  with  ship  hit  estimates  and 
path  assessments  . 


T able  5-5  summarizes  these  data  when  the  rankings  are  projected  from  the  qualitative 
assessments  of  part  III.  This  table  shows  the  same  pattern  of  table  5-4:  subjects  decisions  are 
more  consistent  with  their  estimates  of  ship  hits  than  with  their  qualitative  path  assessments. 

Although  these  data  do  not  completely  preclude  the  existence  of  wholistic  path  schemata, 
they  surely  do  not  support  them  either.  It  seems  likely  that  In  this  experiment  such  schemata  did 
not  form.  It  is  possible  that  with  enough  experience,  people  would  develop  these  schemata. 
There  are  several  reasons  why  they  did  not  form  in  these  experiments,  however. 


First,  during  training  there  was  no  reinforcement  of  patterns.  The  ship  locations  are 
repeated,  but  no  two  pictures  have  the  same  combination  of  ships.  Because  the  pictures  are  so 
varied,  it  would  be  very  difficult  for  subjects  to  remember  previously  seen  patterns. 


training  group 

ranking 
predicted  by 
ahip  eatimate 
only 

ranking 
predicted  by 
feature  statement 
only 

total  number 
of  caaea 

measure  group 

61 

33 

94 

curve  group 

30 

24 

54 

outcome  group 

43 

27 

70 

Table  5-5  Comparison  of  decision  consistency  with  ship  hit  estimates  and 
feature  statements. 


Second,  there  may  be  a  tendency  for  people  to  anchor  on  a  particular  training  method. 
Subjects  are  unable  or  unwilling  to  completely  abandon  the  outcome  calculation  rule  in  favor  of  a 
feature  or  wholistic  approach.  Faced  with  time  pressure  that  prevented  them  from  using  the  exact 
method  taught  in  training,  they  chose  to  approximate  that  method  rather  than  switch  to  an  entirely 
different  method. 

Third,  the  differences  between  the  options  in  the  training  and  test  pictures  are  small. 
Therefore,  the  subjects  may  have  judged  that  any  wholistic  schemata  being  formed  were 
inadequate  to  select  among  the  alternatives,  and  therefore  the  selection  should  be  based  on  the 
"eyeball  and  count"  strategy. 

5.3,7.  Effects  of  training  method 

The  three  different  training  modes,  "measure,"  "curve,"  and  "outcome,"  were  selected  to 
test  different  hypotheses  concerning  the  use  of  schemata  and  other  memory  models.  It  was 
conjectured  that; 

1 :  The  contour  group  would  develop  the  best  mental  contours,  and  would  be  best  at  ship 
hit  estimates  if  these  contours  are  used  for  that  purpose.  In  addition,  subjects  in  this  group  would 
most  easily  see  patterns  of  ships,  and  would  be  most  likely  to  develop  wholistic  schemata. 

2.  The  measure  group  would  pay  the  most  attention  to  Individual  ships.  Because  subjects 
in  this  group  are  given  a  rule  or  guidelines  for  determining  ship  hits,  they  should  have  the  best 
understanding  of  the  factors  important  in  estimating  ship  hits.  It  was  anticipated  that  they  would 
have  the  best  ship  hit  schemata. 
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3.  The  outcome  group  would  have  neither  the  contour  curves  nor  the  measurement  rules 
to  guide  them.  This  group  was  expected  to  perform  less  well  than  the  others. 

The  different  training  conditions  did  help  to  discriminate  among  the  different  menrrary 
models.  The  curve  group  did  not  estimate  ship  hits  better  than  did  the  measure  group,  indicating 
that  mental  contours  separating  the  task  map  into  areas  of  zero,  one,  and  two  hits  are  unlikely  to 
be  used  in  estimating  ship  hits.  None  of  the  groups  seem  to  have  developed  wholistic  schemata. 
As  the  data  in  Tables  5-4  and  5-5  show.  It  is  unclear  that  the  curve  group  showed  more  evidence 
for  these  schemata  than  the  other  groups 

Differences  between  training  groups  in  performance  during  the  test  parts  of  the 
experiments  were  generally  not  significant.  This  is  rK}t  because  all  training  methods  were  equally 
effective,  but  rather  because  subjects  trained  to  criterion.  Performance  differences  were 
minimized  because  the  only  data  analyzed  was  tor  subjects  who  reached  criterion. 

While  it  seems  that  different  training  methods  do  not  cause  different  mental  structures  for 
this  task,  different  methods  of  training  do  influence  the  rate  at  which  formation  for  these 
structures.  The  most  efficient  training  method  was  the  curve  method.  Only  23  subjects  were  run 
to  have  20  reach  criterion.  The  second  most  efficient  training  method  was  the  measuring  method 
in  which  28  subjects  had  to  be  run.  The  least  efficient  training  method  was  the  outcome  method 
in  which  32  subjects  had  to  be  run. 
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6.  GENERAL  DISCUSSION 

The  two  experiments  described  in  this  report  examined  how  memory  interacts  with  outcome 
projection  in  decision  making.  In  both  decision  tasks  subjects  were  taught  analytical  procedures 
for  identifying  the  best  decision  alternative.  These  procedures  were  complex,  requiring  several 
minutes  to  use.  It  was  expected  that  with  experience  subjects  would  develop  reference  data  in 
memory  that  would  enable  them  to  make  good  decisions  when  conditions  would  not  allow  them  to 
use  the  analytical  methods.  During  testing  subjects  were  given  only  a  few  seconds  to  select  the 
best  alternative,  a  time  much  too  brief  to  use  the  precise  analytical  procedures. 

6.1.  Relationship  between  experiments 

The  two  experiments  complemented  each  other.  While  concerned  with  the  same  overall 
issues,  they  focused  on  different  detailed  aspects  of  the  use  in  memoi  /  in  these  decision  making 
tasks.  Table  6-1  summarizes  the  focus  of  the  two  experiments. 


Experiment  1 


Experiment  2 


Use  of  memory  in  task 


Fuzzy  set  schema 
for  ships 


Contours  dividing  areas 
of  0, 1 ,  and  2  ship  hits 


Indicator/  counterindicator 
features 


Eyeball  and  count 
alternative  selection 


Wholistic  path  schemata 


Table  6-1.  Issues  sddressed  by  Experiments  1  and  2. 

Symbol  legend;  ++  Investigated  In  depth;  +  addressed;  -  did  not  address. 

6.2.  Summary  of  results 

Figure  6-1  summarizes  different  hypotheses  concerning  the  use  of  memory  in  selecting  the  best 
action  for  the  decision  tasks  in  these  experiments.  Memory  could  oontribute  to  two  different  parts 


MEMORY  REFERENCE 


JUDGMENTS 


REMEMBERED 

INDIVIDUAL 

INSTANCES 


SCHEMA  FOR  3 
INDIVIDUAL 
SHIP  HITS 


CONTOUR  4 
PLOTS 


INDICATOR/  6 
COUNTER-INDICATOR 
FEATURES 


SCHEMA  FOR 
SHIP  CLUSTERS 


Figure  6-1:  Investigated  memory  structures  that  might 
explain  decision  making  performance. 


of  the  process:  1 )  estimating  the  number  of  hits  from  ships,  and  2)  seiecting  among  the 
alternatives  (straight  path,  curved  path,  or  stay). 


The  data  indicate  that  in  these  tasks  people  employ  both  schemata  and  outcome  calculation 
methods.  They  use  the  fuzzy  set  schemata  of  the  kind  illustrated  in  Figures  3-4  and  3-6  for 
estimating  ship  hits,  and  then  evaluate  the  alternatives  by  summing  these  estimates  of  individual 
ships.  The  subjects  did  not  use  indicator/counterindicator  features  to  bypass  estimating  hits 
along  each  path  when  making  their  decisions.  There  is  highly  suggestive  evidence  that  people 
do  not  have  or  use  contour-like  discriminators  in  estimating  ship  hits.  Finally,  there  is  strong 
evidence  that  these  subjects  did  not  develop  ’wholistic'  path  schemata  for  path  evaluation. 

These  conclusions,  and  the  most  important  data  supporting  them,  are  summarized  in  Tables  6-2 
and  6-3. 


Table  6-2 


REASONS 


Ship  hit  estimates  made  without  measuring 
tools  are  much  better  than  random.  (1 ),  (2) 


Subjects'  estimates  of  hits  from  ships  distant 
from  previously  observed  ships  can  be  as 
accurate  as  their  estimates  of  ships  rrear  those 
of  previously  observed  ships.  (1) 

Subjects’  estimates  of  ship  hits  from  ships 
equidistant  from  target  points  are  equally 
accurate,  whether  ships  were  previously  seen 
or  not.  (1) 

Estimated  number  of  hits  increases  with 
distance  from  origin,  and  decreases  with 
distance  from  path.  (2) 

Contour  plots  can  be  drawn,  but  are  not  drawn 
very  well.  Contour  size  arnf  general  shape  are 
often  Nghly  inaccurate.  (1 )  (2) 

Ship  hit  estimates  predicted  from  subjects' 
drawn  contours  are  usually  significantfy 
different  from  their  estimates  provided  directly 
(2). 

Subjects  provided  with  ground  truth  contour 
curves  during  training  did  not  estimate  ship 
hits  more  accurately  than  subjects  not 
provided  with  these  curves.  (2) 


Summary  o1  conclusions  about  sub)acts'  usa  of  mamory  structura  to 
estimate  hits  from  each  hostile  ship.  Support  attained  in  experiment  1 
denoted  by  (1);  support  attained  in  experiment  2  denoted  by  (2). 


CONCLUSION 

Subjects  developed  memory 
reference  data  to  help  them 
estimate  ship  hits. 

Subjects  use  more  than  just  the 
location  of  remembered  instances 
in  making  their  assessments. 


Subjects  use  schemata  like  those 
in  Figure  3-4  for  estimating  hits 
from  individual  ships.  Schemata 
are  organized  around  the  location 
of  previously  observed  ships. 
They  contain  data  for  estimating 
changes  in  hits  resulting  from 
small  changes  in  ship  location. 

Subjects  do  not  have  accurate 
contour  like  discriminators  that 
separate  the  sets  of  ships  able  to 
score  zero,  one,  and  two  hits. 


CONCLUSION 


REASONS 


1 .  Path  ranking  is  based  on  the  sum  1 .  Consistency  of  path  ranking  with  rankings 

of  estimated  ship  hits.  predicted  from  ship  hits  estimates  is  as  high  as 

path  ranking  test-retest  consistency  (which 
reflects  individual  ship  hit  test-retest 
vevifybility).  (2) 

2.  Subjects  did  not  use  2.  Subjects  provided  with  explicit  features  useful 

indicator/counterindicator  for  path  ranking  did  not  notice  nor  use  these 

features.  features.  (1) 

3.  Subjects  did  not  use  path  schema  3.  Wholistic  assessments  of  path  quality  do  not 

to  rank  paths.  predict  subject's  path  rankings  as  well  as  do 

subject's  ship  hit  estimates.  (2) 

Training  method  that  encourages  wholistic 
pattern  recognKion  did  not  change 
relationship  between  hit  estimates,  path 
quality  rankings,  and  patti  rankings  .  (2) 

Table  6-3  Summary  of  concluslona  about  8ub|ect's  uae  of  memory  atructurea 
to  rank  patha.  Support  attained  in  Experiment  1  denoted  by  (1);  support 
attained  in  Experiment  2  denoted  by  (2). 


6.3.  Generality  of  results 

Earlier  research  by  Engineering  Research  Associates  showed  that  the  fuzzy  schema  nfK>del 
could  explain  a  subject's  evaluation  of  situations  in  terms  of  objective  features  of  the  situation. 

The  subjects  in  those  experiments  were  trained  by  examples  explained  in  terms  of  situation 
features.  In  contrast,  the  experiments  reported  here  sought  to  determine  whether  such 
schemata  would  form  and  be  used  by  subjects  when  they  were  trained  to  evaluate  decision 
alternatives  using  an  analytical  outcome  evaluation  process.  The  data  Indicate  that  subjects  wiH 
develop  and  use  such  schemata,  but  that  these  schemata  may  support  evaluations  used  in  steps 
of  the  analytical  processes  rather  than  directly  supporting  the  overall  process.  These  subjects 
integrated  both  schemata  and  outcome  calculation  in  their  overall  decision  process.  Furthermore, 
the  type  of  schemata  that  developed  did  not  vary  with  training  condition,  although  the  rate  of 
schemata  development  did.  It  seems  likely  to  us  that  these  observations  will  be  true  in  many 
decision  tasks.  In  particular,  it  is  likely  (1 )  that  with  experience  schemata  will  form  to  assist  with  the 
decision  process,  (2)  that  the  the  fuzzy  schemata  observed  here  may  be  the  usual  type  that  forms 
regardless  of  training  method,  and  (3)  that  these  schemata  can  be  Integrated  with  outcome 
calculation  in  various  decision  making  strategies. 


6.3.2.  Typ«  of  schema 
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The  schemata  proposed  to  explain  our  data  have  several  general  properties  that  may 
characterize  schemata  used  in  situation  assessment  and  decision  making: 

1 .  The  schemata  are  organized  around  a  prototype,  a  typical  example  of  a  type  of  situation.  In 
our  experiments,  these  were  the  ships  observed  during  training. 

2.  The  schemata  define  the  features  that  are  relevant  for  assessing  situations  modeled  by  the 
schemata.  For  ship  hit  schemata  of  Experiment  1 ,  proposed  features  are  the  radial  and 
angular  displacements  of  the  new  ship  from  the  previously  seen  "prototype"  ship.  For  ship 
hit  schemata  of  Experiment  2,  proposed  features  are  the  vertical  and  horizontal 
displacements  of  the  new  ship. 

3.  The  schemata  contain  "feature  evaluation  data"  that  enable  people  to  evaluate  the 
significance  of  differences  between  an  observed  situation  and  the  typical  prototype 
situation. 

4.  The  schemata  associate  judgments  with  situations  modeled  by  the  schemata.  For  ship  hit 
schemata  of  Experiment  1  and  2,  these  are  the  number  of  hits  by  a  hostile  ship  at  a  target 
point  and  along  the  paths. 

In  addition  to  the  properties  listed  above,  decision  oriented  schemata  may  have  the 
following  characteristics: 

1 .  The  feature  evaluation  data  may  be  organized  as  schemata  themselves.  Thus,  the  overall 
schemata  for  evaluating  a  situation  may  more  accurately  be  regarded  as  a  network  of 
embedded  schemata.  This  property  of  schemata  have  been  documented  by  numerous 
investigators.  (Rumelhart,  t981;  Bower,  Black,  and  Turner,  1979) 

2.  The  features  may  be  more  abstract.  The  feature  identification  layer  of  the  schemata  may 
identify  functional  as  well  as  physical  properties  of  situations  relevant  to  situation 
assessment. 

3 .  The  schemata  may  be  "built  on  the  fly ."  Schemata  may  not  exist  in  long  term  memory. 
Rather  they  may  be  developed  as  needed  from  data  in  long  temn  monx>ry.  The  schemata 
then  function  to  evaluate  observed  situations  by  comparing  the  properties  of  observed 
situations  with  expectations  specified  by  the  schemata. 

The  first  two  characteristics  of  schemata  were  observed  in  the  earlier  Engineering  Research 
associates  investigations.  That  last  property  is  hypothesized  to  conserve  mennory  requirements 
in  long  term  memory.  The  data  on  conjunctive  fallacy  (Tversky  and  Kahneman,1983)  arKi  on 
jurors'  evidence  evaluation  (Pennington  and  Hastie,  1986)  also  suggest  this  mechanism.  It  is  not 
known  how  schemata  are  buHt  in  order  to  process  a  decision  task. 

6.3.3.  Effect  of  training  and  experience 

We  also  conjecture  that  schemata  are  the  foundation  for  expert  decision  making.  We 
believe  that  with  experience  people  will  develop  schemata  that  enable  them  to  identify  high 
quality  alternatives  directly  from  the  characteristics  of  the  situation.  Such  schemata  are  proposed 
to  account  for  the  high  quality  of  alternatives  considered  by  chess  masters  (Chase  arKl  Slnron, 
1973). 
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Although  the  ship  hit  schemata  were  not  examined  in  detail  in  the  second  experinient,  the 
data  from  this  experiment  suggest  the  same  type  of  fuzzy  schemata  will  develop  with  each  of  the 
three  training  conditions.  If  so,  then  such  schemata  may  represent  a  preferred  or  "natural" 
memory  structure  for  schema-based  decision  making.  Although  training  method  may  not  affect 
the  type  of  schemata  that  eventually  develop,  training  method  is  likely  to  affect  the  rate  at  which 
they  form.  Given  the  characterstics  of  these  schemata,  they  might  develop  most  easily  given 
training  conducted  in  the  following  manner: 

1 .  The  training  begins  with  a  description  of  typical  situations  in  which  various  judgments  and 
actions  are  warranted. 

2.  Training  continues  with  theory.  The  theory  explains  why  particular  kinds  of  actions  are 
recommended  for  different  kinds  of  situations.  The  theory  may  include  outcome 
calculations  and  may  indude  causal  relationships  linking  situation  features  with  actions 
outcomes. 

3.  Another  phase  of  training  provides  examples  of  situations  that  vary  from  the  typical 
situations  described  earlier.  Each  of  these  variations  is  related  to  a  schema  feature.  The 
training  emphasizes  how  the  variation  affects  the  judgments  that  should  be  made  and  the 
actions  that  should  be  taken.  The  training  should  also  specify  in  terms  of  the  theory  why 
such  variations  have  their  effects. 

6.3.4.  Hybrid  decision  strategies 

The  decision  making  strategy  observed  in  these  experiments  contained  elements  of  both 
schema-based  and  outcome  calculation  decision  making.  Such  hybrid  strategies  are  likely  to  be 
common.  The  extent  to  which  these  two  elements  arises  in  any  decision  will  depend  on  the 
characteristics  of  the  task,  the  familiarity  of  the  decision  maker  with  the  task,  the  severity  of 
consequences  for  bad  decisions,  and  preferred  decision  modes  specific  to  the  individual 
decision  maker. 

Because  many  decisions  are  likely  to  be  based  on  both  schemata  and  outcome  calculation, 
and  because  the  roles  of  these  two  processes  will  depends  on  numerous  task  and  decision 
maker  characteristics,  it  Is  not  meaningful  to  €isk  whether  a  decision  was  based  solely  on  schemata 
or  whether  it  was  based  on  outcome  calculation,  it  will  often  be  based  on  both.  It  Is  meaningful, 
however,  to  ask  under  what  conditions  these  different  processes  are  used,  how  they  interact 
when  both  contribute  to  a  decision,  and  how  the  processes  affect  the  quality  of  decisions. 

6.4.  Implications  for  distributed  decision  making 

Distributed  decision  making  depends  on  different  people  making  similar  interpretations  of 
situations  and  having  a  common  understanding  of  the  actions  to  be  taken  in  these  situations.  It 
also  depends  on  the  decision  makers  identifying  information  which  is  most  important  to  seek  and 
communicate. 

These  research  results  suggest  a  particular  kind  of  memory  organization  may  mediate 
decision  making  in  which  situation  recognition  is  Important.  Training  and  information  presentation 
methods  designed  to  interact  with  memory  organized  in  this  fashion  may  enable  people  to  team 
more  easily  how  to  interpret  situations.  These  methods,  if  applied  to  groups  of  people,  may 


enable  them  to  interpret  situations  more  uniformly.  Schema  motivated  training  procedures  were 
described  earlier. 

The  schema  theory  also  suggests  information  presentation  methods.  These  methods 
make  explicit  in  a  display  the  schema  in  memory  that  are  used  for  situation  assessment.  These 
displays  show  two  types  of  information:  schema  derived  situation  reference  data  and  data 
describing  a  current  situation.  The  schema  derived  information  emphasizes  those  situation 
features  identified  by  the  schema  as  relevant  for  assessing  a  situation.  It  also  shows  the 
significant  deviations  between  the  observed  situation  and  the  situation  prototype. 

The  schema  displays  help  identify  important  information  to  seek  and  communicate  by 
indicating  which  currently  ambiguous  situation  features  might,  when  daiified,  significantly  affect 
the  situation  interpretation. 
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APPENDIX  A 


Sample  picture  with  single  ship  outside  curved  path  (Experiment  1). 


overlapping  submarine  areas  (Experiment  1). 
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